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Intelligent  Tutoring  Systems:  Past,  Present,  and  Future 

SUMMARY 

In  this  paper,  we  address  many  aspects  of  Intelligem  Tutoring  Systems  (ITS)  in  our 
smdi  for  answers  to  foe  ftdlowing  main  questitms;  (a)  What  are  foe  precursors  of  ITS?  (b)  What 
does  the  teim  metm?  (c)  What  are  some  important  milestones  and  issues  across  the  20+  year 
hisimy  of  ITS,  (<0  What  is  foe  status  of  ITS  evalumicHis,  and  (e)  What  is  the  future  of  ITS?  We 
stait  wifo  an  lUstorical  perqtective. 

PRECXJRSORS  OF  ITS 

Eariv.  Mcchinkal  Sysinns 

Ouules  Babbage  (eariy  1800s),  is  typically  credited  wifo  being  the  first  to  envisitm  a 
mnlti-poipoae  cooqMter.  He  dreamed  of  creating  an  all-purpose  madiineuliidi  he  called  foe 
"analytic  engine."  However,  because  of  foe  tedmological  omstiaints  of  foe  time,  he  was  never 
able  to  build  bis  dream,  alfoougb  he  did  succeed  in  building  a  difference  engine,  an  automatic 
(mechanical)  means  of  calculating  togaritfun  taUes. 

Tie  notion  of  using  "intelligent  machines"  for  teadiing  purposes  can  be  traced  back  to 
1926  when  Pressey  built  an  instructional  machine  teeming  wifo  multiple-cfaoice  questimis  and 
answers  submitted  by  foe  teadier.  It  delivered  qpwstions,foen  provided  immediate  feedback  to 
eadi  learner 

The  somesriiat  astounding  way  in  iriiidi  foe  functioning  of  foe  qiparatus  seems 
to  fit  in  wifo  foe  so-called  laws  of  learning’ deserves  mention  in  this  connection. 

The  law  of  recency' operates  to  estaUifo  foe  correa  answer  in  the  mind  of  foe 
subject,  since  it  is  always  foe  tor  answer  which  is  the  ri^  one.  Ihelawof 
fie^Mncy*  also  cooperates;  by  chance  the  ri^  response  tends  to  be  made  most 
often,  since  it  is  foe  or/k  reqxHise  by  whidi  foe  subject  can  go  on  to  foe  next 
questitm.  FUrfoer,  wifo  the  addition  of  a  simile  attadunent  foe  apparatus  will 
pieaent  foe  subject  wifo  a  piece  of  candy  or  other  reward  upon  his  makii^  any 
given  score  for  which  foe  eqrerimentermay  have  set  the  device;  that  is  foe  law 
of  effect*  also  can  be  made,  automatically,  to  aid  in  foe  estaUifoing  of  the  right 
answer  (Pressey,  1926,  p.  375). 

While  the  above  system  was  definitdy  clever  for  its  time,  it  could  not  be  omstrued  as 
fai^niflwir  as  it  was  mechanically  set  wifo  pie-qiecified  questions  and  answers.  So,  alfoough  it 
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was  inflexiUe,  this  system  did  incoiporate  contemporaiy  learning  theories  and  pedagogical 
stiategies  into  its  design  (e.g.,  giving  out  candy  for  conect  responses). 

Oeneral-puipose  digital  omputers  arose  in  the  mid-1900s,  paving  the  way  for  truly 
(artificially)  intelligent  machines.  Basically,  these  computeis  consisted  of  a  numerical  central 
processor  a^Kxse  medianism  was  electnaiic,  not  mechanical,  and  based  on  a  binaiy,  not  decimal, 
syston.  They  were  also  characterized  by  having  a  built-in  aUlity  to  make  logical  decisimis.  and 
a  built-in  device  for  easy  storage  and  manipulation  of  data. 

During  this  period  of  computer  infancy,  Ahm  Ttiring  (1912-1954,  British  mathonatician 
and  logician)  provided  a  major  link  between  these  modem,  digital  computing  systems  and 
drinking.  He  described  a  cmnputing  system  capeiAe  of  not  only  "number  crunching"  but 
symbolic  mairipulaticm  as  well.  He  also  developed  what  is  now  known  as  the  "Triring  test,"  a 
moms  of  determirring  a  machine's  intelligence.  The  test  consists  of  an  individual  asking 
questions,  in  real-time,  of  both  a  human  and  cmnputer.  The  interrogator  attempts,  in  any  way 
possitrie,  to  figure  out  which  is  which  via  ccmveisations  over  the  communication  lirrks.  The 
Triring  test  has  particular  relevance  to  intelligent  tutoring  systems.  Hie  core  concept  bdrind  die 
test  is  whedier  a  reasoiuririe  persm  can  distinguish  between  a  computer  and  a  person  based  soldy 
on  their  respective  responses  to  whatever  questions  or  statements  the  inteiKogator  renders.  Titus, 
for  a  computer  to  pass  die  test,  it  would  need  to  communicate  like  a  Iniman,  which  is  a  nontrivial 
goal.  This  line  of  inquiry  has  challenged  and  occuiriedreseardhers  for  die  past  2(>f  years,  and 
oontiiioes  to  {riay  a  prominent  role  in  the  development  of  ITS  (see  Menill,  Reiser,  Ranney,  and 
Traflon,  1992).  Odier  communication-ielaled  research  includes  devising  knowledge  structuring 
and  hypertext  tedmiques  widrin  ITS  to  provide  answers  to  the  many  possiUe  questions  that 
studems  could  pose  to  the  system.  So.  the  success  of  dris  ITS  enterprise  ready  can  be  measured 
in  a  way  that  is  similar  to  die  Triiing  test:  How  well  can  the  ITS  conanunicate?  We  should  point 
out,  however,  that  die  goal  of  ITS  is  to  communicate  its  embedded  knowledge  effectively,  not 
necessarily  in  an  identical  manneras  human  leacfatgx.  In  fact,  some  teachers  have  great  difficulty 
adrieving  the  effective  communication  goal  diemsdves. 

Cfoncunent  widi  die  gradual  emergence  of  computers  tm  the  scene  (area  19S0s), 
educational  psyduriogists  begm  reporting  in  the  literature  that  careiriUy-designed,  individualized 
tutoring  produces  the  best  leanring  for  the  most  peoirie  (e.g..  Bloom,  1956;  CarroU,  1963; 
Crowder,  1959;  Glaser,  1976;  Skhmo’,  1957).  Thus,  it  was  quite  a  natural  devdopment  to  apfriy 
computers  to  die  task  of  indivittaializedteachiiig.  Bom  the  1970s  to  the  present,  ITS  have  been 
heralded  as  die  most  promising  qipioadi  to  delivering  such  individualized  instruction  (e.g.. 
Burton  &  Brown,  1982;  Lewis,  McArdiur,  Stasz  &  Zmuidzinas,  1990;  Shute  &  Regian,  1990; 
Sleemim  &  Brown.  1982;  Wenger,  1987;  Woolf,  1988;  Yazdani  &  Lawler,  1986).  Well  now 
review  uriutt  led  to  the  development  of  "irrielligait"  computerized  instnretimL 
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In  die  eaily  1960s,  {Mognunmed  instnicdon  <F1)  was  educationally  fashionaUe.  This 
kind  of  pedagogy  idaled  to  any  structured,  goal-oriented  instnictitHL  According  to  Bunderson 
(1970),  n  required  the  program  designer  to  q)ecify  iiqmt  and  ouqmt  in  terms  of  entering  dcilis 
and  terminal  behaviors  of  the  learner.  In  performing  a  task  analysis,  the  designer  determined  die 
sub-problems  or  component  behaviors,  as  well  as  dieir  relationships.  As  learners  were  led 
dircMi^  the  proUems  in  the  curriculum  (lock-stqi),  overt  responses  were  obtained  at  every  step; 
incorrect  responses  were  immediately  corrected,  and  learners  were  always  informed  of  their 
soludrm  accuracy  before  moving  (Ml  to  some  other  cmitent  area.  Most  supporters  of  die  Pi 
tecdmcdogy  strongly  believed  diat  it  would  enhance  learning,  particulaily  for  low  aptitude 
individuals.  However,  evidence  siqiporting  this  belief  was  underwhelming  (see  Cronbach  & 
Snow.  1981). 

In  general,  PI  refers  to  any  InstnictkMial  methodology  that  utilizes  a  systematic  approach 
to  proUem  decomposititMi  and  teadiing  (e.g.,  Briggs.  Campeau,  Gagnd,  &  May,  1967;  Gagn6, 
196S).  That  PI  results  in  a  computer  program,  known  as  conq)uter-assisledinstiucti(Mi(CAI  or 
computer-based  training,  CBT).  Some  similarities  between  PI  and  CAI  are  that  both  have  well- 
defined  curricula  and  branching  routines  (intrinsic  Inandiing  for  n,  (XMiditional  branching  for 
CAI).  A  major  distinction  between  die  two  is  that  CAI  is  administeied  (Ml  a  conqniter. 

Computer-assisted  instructi(m  also  evolved  frcMU  Skhmerian  stimubis-ie^nse 
psychology,  "...die  student’s  le^xMise  serves  prhnarily  as  a  means  of  determining  whether  die 
communicatkMi  process  has  been  effective  and  at  die  same  time  allows  qipiqMiale  conective 
action  to  be  taken"  (Crowder,  19S9).  In  odier  words,  at  every  point  in  the  curriculum,  the 
(XMnputer  program  evaluates  ^ledier  the  student’s  answer  is  right  or  wrong  and  then  moves  the 
student  to  the  proper  padL  Built-in  remediaticMi  loops  tutor  students  who  are  attempting  to 
answer  a  (}uesti(m  incortecdy.  If  leatnets  answo-  conecdy.  they  are  moved  ahead  in  die 
curriculum.  Hgure  1  illustrates  a  typical  flow  of  events  in  CAI. 
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Com putcr-Asiiitcd  Imtruction 


Stsft 


BaiM*  Frocraa  aetioa*:  BIHaifla.*  Canaad  protram  kaowtadt* 

Figure  1 


TIk  teacher  coDSliucteaUbniidiiiig  in  the  program,  ahead  of  time.  Tbe  nonnal  CAI 
piocediire  presents  some  material  to  be  teamed,  followed  by  a  problem  to  be  solved  that 
represents  a  subset  of  tiie  curriculum.  ProUem  solutitm  tests  the  leaniei's  acquisitimi  of  the 
knmiriedge  or  skill  being  instnided  at  that  time.  The  student's  answer  is  compared  to  tiie  correct 
answer,  then  the  cmqrater  gives  qipropriate  feedback.  If  tiie  answer  is  correct,  a  new  (HoUem  is 
selected  and  piesoited,  but  if  tiie  student  answers  incorrectly,  lemediatitm  is  invdced  that  reviews 
tiie  earlier  material,  presents  sinqder  (Mobtems  tiurt  graduate  to  the  depth  of  tiie  original  material, 
andsofortiL  RemediaticHi  usually  requires  seme  attemiH  to  rind  the  source  of  the  error  and  to 
treat  it  qwdaily. 

As  can  be  seen  in  the  figure,  there  are  several  {daces  i^re  this  simple  model  may  be 
esqwnded  to  create  m(»e  flexiUlity  and,  hence,  render  it  adaptive  to  individual  learners.  Fbr 
instance,  various  mastery  criteria  can  be  imiiosed,  where  subjects  have  to  answer  a  certain 
pre^xntion  of  items  correctly  before  moving  on.  Failure  to  readi  criterion  would  force  the 
studrat  bade  into  remediation  mode  (see  "If  Incorrect”  Inanch)  where  a  differmt  problem  is 
{xesented,  ratiier  tiian  the  {HoUem  that  caused  the  error. 
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Intelligent  Cttmiiuter-Assigied  Instnicdon 

To  distinguish  between  simple  versus  more  adaptive  CAl  (i.e.,  "intelligent"  computer- 
assisted  instiuctioii,  ICAI),  Wenger  (1987)  pointed  out  that  actually  there  is  no  exi^cit 
demarcation  between  the  two.  Instead,  there's  a  omtinuum,  from  linear  CAl,  to  more  cmnplex 
branching  CAl,  to  elementary  ICAI,  to  autonmnous  (or  stand-alone)  ICAI.  Ihis  continuum  is 
often  misconstrued  as  refMesenting  a  woise-to-better  fxogressioiL  Yet,  for  scnne  learning 
situations  and  for  some  curricula,  using  fancy  programming  techniques  may  be  like  using  a 
shotgun  to  kill  a  fly.  If  a  drill-and-practice  envirmunent  is  all  that  is  required  to  attain  a 
particular  instructional  goal,  dien  that's  what  should  be  used. 

Suppose  you  wanted  to  build  a  computerized  instruciirmal  system  to  help  second  graders 
learn  double-digit  addidoiL  If  Audent  A  answered  die  following  two  problems  as:  22  +  39  » 

«id  46  4-  37  s  JS2.  you'd  surmise  (with  a  fair  amount  of  confidence)  that  A  understood,  and  could 
successfully  iq)|dy,  the  "carrying  procedure."  But  consider  some  other  responses.  Student  B 
answers  the  same  proUems  with  51  and  73,  student  C  answers  with  161  and  203,  and  student  D 
answers  widi  61  and  85.  Simfde  CAl  systems  may  be  incapable  of  differentiating  these  incorrect 
stdudons,  and  remediatioa  would  require  all  three  students  to  re-do  die  specific  unit  of 
instructicML  But  a  big  protdem  widi  diis  approach  is  diat  typically,  diere  is  litde  difference 
between  die  remedial  and  original  instructirxL  Hiat  means  that  a  student)^  didn't  get  it  right 
die  first  time,  may  not  get  it  ri^  die  next  time  if  the  same  instracdmi  and  similar  proUons  are 
used. 

A  more  sensitive  (or  intelligent)  respcmse  by  die  syaem  would  be  to  diagnose/dassify 
B's  answer  as  a  ftdlure  to  cany  a  one  to  the  tens  odumn,  Cs  answer  as  the  incorrect  adding  of  the 
tmes  column  result  (11  and  13)  to  the  tens  column,  and  B's  as  a  probaUe  computational  error  in 
the  second  proUan  (mistakenly  adcUng  6  4- 7  s  15  iiffitead  of  13).  An  intelligent  system  would 
remediate  by  specifically  addressing  each  of  die  three  qualitatively  different  errors. 

Aittfiriai  Iiirriliyence  and  CcHmitive  Psvchnlogv.  How  can  a  computer  system  be 
progtamined  to  perform  intelligently?  lUs  question  drives  die  eminrical  and  engineering 
researdi  in  a  field  called  artificial  intelligence  (AI).  The  simplest  definition  is  diat,  "Artificial 
intelligence  is  the  Study  of  mental  faculties  dumi^  the  use  of  computational  models."  (C2iamiak 
&  McDermott,  1985,  p.  6).  One  of  the  main  objectives  of  AI  is  to  design  and  development  of 
(xmipiiter  systems  that  can  stdve  the  same  kinds  of  activities  that  we  deem  intelligent  (e.g., 
scdving  a  math  proUem  like  die  one  illustrated  above,  understanding  natural  language, 
programming  a  conqmter  to  perftnm  some  functioiKs),  maneuvering  an  aircraft  through 
obstacles,  planning  a  wedding  reception,  and  so  fordi).  There  are  far  too  many  AI  rqifdications  to 
deUneate  in  this  duster.  For  our  purposes,  Alteclmiquestelevam  to  ITS  include  those  dealing 
with  the  efficient  repesentation,  storage,  md  retrieval  of  knowledge  (i.e.,  a  large  collection  of 
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fiKts  and  skins-correct  and  buggy  versions),  as  well  as  the  effective  communication  of  that 
infonnadmi.  In  addition,  AI  techniques  can  include  inductive  and  deductive  reasoning  processes 
diat  allow  a  system  to  access  its  own  database  to  derive  novel  (i.e.,  not  programmed)  answers  to 
leamers'  queries. 

Cognitive  psychology  also  provides  part  of  the  answer  to  the  question  of  how  to  get  a 
computer  to  behave  intelligently  by  examining  issues  related  to  die  representation  and 
organization  of  knowledge  types  in  human  memory.  Research  in  this  area  provides  detailed 
structural  specifications  for  implementation  in  intelligent  computer  programs.  Cognitive 
psychology  also  addresses  the  nature  of  errois,  a  critical  feature  in  the  design  of  intelligent 
systems  to  assist  learners  during  the  learning  process. 

Thfi  NatiiBft  of  Errors.  The  idea  that  shidents  and  trainees  make  mistakes  diat  have  to  be 
corrected  is  fiindamental  to  teaching  and  learning.  Something  so  fundamental  ought  to  be 
strongly  resistant  to  change,  so  it  is  really  quite  surprising  how  the  ittea  of  a  mistake  or  error  has 
undergone  radical  change  over  the  past  two  decades  of  ITS  development  The  traditional  view  of 
errors  encompassed  many  kinds:  fiom  inexidicaide  accidents,  to  deliberate  inaccuracies;  but  die 
most  widely  held  view  was  that  remedial  errors  stemmed  from  inaccurate  or  insufficient 
knowledge.  Remediation  then  corrected  the  mistake  by  providing  the  correct  knowledge  or 
overriding  the  inaccuracy.  The  first  major  shift  that  occurred  in  this  vtew  began  with  the 
developmoit  of  a  theoretical  posititm  dial  errors  arose  because  of  complex  organizations  in 
knowledge  structures  that  were  not  wrong,  in  the  traditional  sense,  but  represoited  the  best  a 
student  could  have  at  that  stage  of  cognitive  develqrment  These  developmentally  apimqrriate 
knowledge  structures  were  called  misconceptions,  and  diey  were  soon  aiudyzed  in  a  broad  range 
of  scimices  (e.g.,  Aristotelian  versus  Newtonian  [diysics,  studies  of  heat  and  temperature)  and 
inactical  training  environments  (autmnotdle  repair,  radar  maintmance). 

This  view  of  error  was  explicated  in  great  detail  in  a  series  of  analyses  and  experiments 
by  Barbara  White  and  Jtdm  Fredetiksen  (1987)  in  dwir  QUEST  system  for  analyzing  levels  of 
understanding  of  electrical  functioning  into  graduated  mental  models.  Their  analyses  were 
actually  imidemented  as  qualitative  models  of  the  electrical  activity  in  automobile  ignition 
dicuits.  Simple  models,  or  models  tiiat  occur  develt^entally  eariy  in  the  growth  of 
knowledge,  were  not  only  inotmiplete,  ttiey  were  wrong  or  inconsistent  in  basic  ways.  They 
could  not  easily  be  transformed  into  more  complete  models.  Yet,  the  simple  models  effectively 
captured  the  knowledge  of  novices  as  they  moved  on  the  road  to  expertise,  so  it  is  not  dear  if 
tiiese  models  could  have  been  imimoved  at  that  stage  of  development  Thus,  it  rqrpeared  that 
error  or  incmisistency  was  necessary  in  the  growth  of  knowledge. 

As  they  demonstrated,  it  took  a  great  deal  of  effort  to  conduct  error  analysis  with 
sufficient  sct^  and  detail  to  be  aUe  to  arrive  at  such  complete  models.  It  is  perhaps  for  this 
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reason  tiiat  no  odierexam{^  comes  close  to  dui^cating  their  feat  Yet  the  intdlectual 
imidications  of  graduated  mental  models  as  the  basis  for  misctmceptitms  and  error  is  stunningly 
qiparent  for  whoever  next  decides  to  pidt  up  the  challenge  and  analyze  knowledge  stnictures  into 
sudi  progressive  systems. 

An  alternate  concepticm  of  error  foat  has  developed  contemporaneously  with  the 
miscoiKepd<m  literature,  is  that  of  a  buggy  algorithm.  Work  in  this  area  began  with  Burton  and 
Brown's  semitud  simulation-How  Ihe  West  Was  Wmi-where  certain  strategic  and  algorithmic 
bugs  were  identified  in  studoit  [day.  A  specific  program  was  written,  DEBUGGY,  ttiat 
attempted  to  identify  and  remediate  diese  bugs  (Brown  &  Burton,  1978;  Burton,  1982).  Unlike 
the  work  tm  misomcqrtions  and  graduated  mental  models,  bugs  were  simpler  deccmstructkHis  in 
smaller  semantic  networks  of  skills. 

Ihis  analysis  of  errors  has  had  a  productive  lire  of  its  own  in  the  woru  of  Soloway 
(catalogs  of  bugs,  Johnson  &  Soloway,  1984),  Sleeman  (mal-rules,  Sleeman,  1987),  and 
VanLefan  (impasses,  VanLehn,  1990).  It  continues  strcmgly  in  the  model-tracing  technology  of 
J(dm  Anderstm's  various  tutors  (e.g.,  Anderstm,  1993)  where  bug  catalogs  or  lists  of  errors  are 
embedded  in  ^wcificpioductkMi-system  rules  diat  manage  all  interactions  between  the  student 
andtutor.  Anderson  has  proclaimed  a  much  broader  view  to  encompass  not  tmly  errors,  but  all 
cognitive  skills.  His  position  is,  simfdy  stated,  tiut  cognitive  skills  are  realized  by  production 
rules.  Not  otdy  errors,  but  all  skills,  are  decomposaUe  into  unitary  rules  that  fit  into  a  grand 
cognitive  architectitre  dominated  by  production  rules. 

VanLehn's  work  rm  impasses  extends  tills  boggy  crmceptirm  of  errors  by  analyzing  the 
ways  these  errors  are  generated  (VanLehn,  1S190).  Oversunftiifying  his  analysis  someuiut, 
VanLehn's  framework  cm  be  described  by  saying  timt  bogs  are  the  result  of  unsuccessful 
attempts  to  extend  existing  rules  to  ap|tiy  to  novel  situations  (repairs).  Ihese  repairs  cm  be 
modeled  and  predicted  by  impasse  tiieory  to  predict  students  bugs  and  proUem  solving.  Usually 
the  rqiaits  are  simftie  actions,  like  removing  m  actitm  stqi  in  the  production  roles,  substituting 
m  operator,  or  deleting  a  variaUe. 

The  final  view  of  errors  that  has  evolved  alot%  with  ITS  sees  the  error  as  a  result  of 
insufficient  support  given  to  the  student  When  a  student  learns  a  new  skill  or  body  of 
knowledge,  it  is  timnigh  the  support  of  teachers,  students,  or  otiier  parts  of  the  environment 
This  environment  acts  as  a  general  scaffolding  to  strengthen  the  midents  first  new  dulls  or 
knondedge  structures  (Palirxsar  and  Brown,  1984).  It  also  fnovides  the  context  tiiat  makes  the 
drills  or  krxmledge  meaningful.  Some  of  this  scaffolding  lies  literally  in  tire  minds  of  the  other 
students  or  teadrers,  or  more  precisely,  between  the  minds  of  everyone.  As  a  kind  of  social 
giDiq>  tiiink,  the  ideas  and  scaffolding  are  part  of  tire  total  situation  (Brown,  (foUins,  &  Duguid, 
1989)  and  so  it  has  been  called  siwated  cognition.  If  the  envinmment  is  literally  part  of  the  skills 


and  knowledge,  then  dunging  it  atuuptly  can  achudly  change  student  thinking  and  lead  directly 
toenois. 

Ihis  fiasdnating  reseaidi  related  to  dififerait  kinds  of  enors  owes  its  existence  directly  to 
the  practical  and  theoretical  developments  that  ITS  have  spawned.  All  have  real  import  for  the 
design  of  instnKXirm,  but  at  the  moment,  they  are  still  very  distant  from  each  (Hirer  and  show  no 
real  signs  of  omvergirig  imo  a  common  theoretical  hamework. 

Summary.  Brandling  is  a  fundamental  aspect  of  PI,  CAI,  and  ICAI.  It  recognizes  the 
fact  that  knowledge  is  interrelated  in  many  complex  ways,  and  there  may  be  multiple  good  paths 
through  the  cuniculum.  AI  programming  techniques  empower  the  computer  to  manifest 
intelligence  by  goir^  beyond  what’s  explicitly  {Hogrammed,  understanding  student  irqxits,  and 
^Derating  raticmal  re^pcMises  based  on  reasmiing  fimn  the  inputs  and  die  system's  own  database. 

In  the  examine  just  {Movided,  prior  to  teaching  double-digit  additicHi,  the  system  could 
first  ascertain  if  the  learner  was  skilled  (to  die  point  of  automaddty)  with  single-digit  additicni, 
drilling  die  learner  across  a  variety  of  proUems,  noting  accuracy  and  latency  for  each  soludcm. 
Subsecpiendy,  it  may  be  effective  to  introduce  (a)  douUe-digit  addition  without  the  carrying 
procedure  (23  41),  (b)  single-  to  douUe-digit  additicm  (5  +  32),  or  (c)  single-digit  addition  to 
10  (7  4- 10).  Each  of  these  curriculum  elements  is  warranted,  and  some  are-easierto  graqi  than 
(Hhers.  However,  for  more  com|dex  knowledge  domains,  sudi  as  history,  or  the  scientific  debate 
over  the  extinctimi  of  dinosaurs,  die  comidexity  of  alternatives  is  beyond  enumeration.  And  it  is 
the  complexity  of  this  branching  that  really  provides  a  ({ualitative  break  between  older  forms  of 
PI  and  CAI  and  newer  ITS.  NcHrmly  is  the  branching  in  ITS  oonqilex,  it  is  algorithmic  and  not 
enumerated,  pre-defined,  or  hand-crafied.  With  diis  qualitative  increase  in  complexity  comes  a 
fiexilHlity  of  interaction  and  potential  forcwnmunication  duu,  better  dum  anything  else  b^re, 
begins  to  qualify  for  the  word  intelligent 

Another  aspect  of  cmnputer  intelligence  deals  widi  the  identificati(m  and  temediati(Hr  of 
errors  (bugs)  in  aleamer*s  knowledge  structure  or  performance.  Hie  simide  illustrati(Hi  with  four 
hypodietical  studoits  shows  die  possitrie  power  of  adding  AI  to  instiucticxial  software  that  can 
recognize  bugs  ormisconcqHions  via:  (a)  a  bug  catalog  diat  specifically  recognizes  eadi  mistake 
(e.g.,  J(dms(m  &  Scdoway,  1984),  (b)  a  set  of  mal-rules  that  define  the  kinds  of  mistakes  possiUe 
with  diis  set  of  (HoUems  (e.g.,  Sleeman,  1987),  or  (c)  a  set  of  {Hoduction  rules  diat  specifically 
anticipate  all  alternative  proUem  solutions  and  can  respcmd  to  each  (Hie  (e.g.,  Anderson,  1993; 
VmLdm,  1990).  Eacdi  of  tiiese  will  be  discussed  in  more  detail  in  the  sectkm  of  this  chrqHer 
(Mitlinir^  the  204  year  history  of  ITS.  I^rst,  we  need  to  (^rationalize  some  terms. 
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INTELLIGENT  TUTORING  SYSTEMS  DEFINED 
While  many  reseaichen  in  Itie  field  view  ICAI  and  ITS  as  inteichangeable  designations, 
we  make  a  subtle  distinctkm  between  the  two:  ITS  represent  a  more  q)ecific  type  of  IGM,  due  to 
die  attributes  discussed  betow. 

Early  Specifications  of  ITS 

An  early  outUne  of  ITS  requirements  was  presented  by  Hartley  and  Sleeman  (1973). 
They  argued  that  ITS  must  possess:  (a)  knowledge  of  die  domain  (expert  model),  (b)  knowledge 
of  the  learner  (student  model),  and  (c)  knowledge  of  teaching  strategies  (tutor).  It  is  interesting 
to  note  diat  this  shnjde  list  has  not  dianged  in  more  tlum  20  years  (see  Lajoie&  Derry,  1993; 
Poison  &  Richaidson.  1988;  Psotka,  Massey,  and  Mutter,  1988;  Regian  &  Shute,  1992;  and 
Sleeman  &  Brown.  1982). 

All  of  diis  computer-resident  knowled^  marks  a  radical  diifl  finom  eariier  "knowledge- 
fiee"  CAI  routines.  Furdiermore,  the  aUlity  to  diagnose  errors  and  tailor  remediatitm  based  on 
the  dit^nosis  rqnesentsakey  difference  between  K}AI  and  CAI.  Figure  2  illustrates  these 
knowledge  components  and  dieirreladons  within  a  genmic  ITS.  Each  of  diese  ITS  components 
will  be  discussed,  in  turn. 


Hgum  2<AdapM  (tom  Kylonen  &  Stwto.  1969 


A  Student  learns  fiom  an  ITS  primarily  by  solving  proUems-<mes  that  are  ^ipn^ately 
selecled  or  tailor-made-  diat  serve  as  good  learning  ejqreriences  for  that  student  The  system 
starts  by  assessing  what  the  student  already  knows,  the  student  model.  The  system  omcurrently 
must  omsider  \iriun  the  student  needs  to  know,  the  curriculum  (also  known  as  the  dmnain 
expert),  finally,  the  system  must  decide  what  curriculum  element  (unit  of  instruction)  ought  to 
be  instructed  next,  and  how  it  shall  be  presented,  die  tutor  (or  inherent  teaching  strategy).  Fmn 
all  of  diese  considerations,  the  system  selects,  or  generates,  a  problem,  then  either  works  out  a 
soludon  to  die  proMem  (via  the  domain  expert),  or  retrieves  a  prepared  solution.  The  ITS  then 
compares  its  solution,  in  real-time,  to  the  one  die  studem  has  [aepared  and  performs  a  diagnosis 
based  rni  differences  between  the  two. 

Feedback  is  offered  by  the  ITS  based  (»i  the  student-advisor  consideratkais  such  as  how 
Irnig  it*s  been  since  feecDiack  was  last  (Movided,  whedier  die  studem  already  received  some 
particular  advice,  and  so  on.  After  the  feedback  loop,  the  program  updates  the  student  skills 
model  (a  record  of  nriiat  die  studem  knows  and  doesn't  know)  and  increments  learning  progress 
indicators.  These  updatfog  activities  modily  the  studem  model,  and  the  entire  cyde  is  rqieated, 
starting  with  selecting  or  generating  a  new  proUem. 

Not  an  ITS  indude  these  conqxments,  and  die  prddmn-test-feedbadc  cyde  does  not 
adequately  cliaiacterfare  aU  systems.  However,  diis  generic  depiction  does  describe  many  cun«.  x 
ITS.  Alternative  im|dementations  exist,  rqtiesaiting  Gtmcqttual  as  weU  as  {xactical  differences 
in  dieir  design.  Fbr  examine,  the  standard  ^iproadi  to  building  a  studem  modd  involves 
representing  emerging  learner  knowle^  and  skills.  The  computer  reqxmds  to  iqtdated 
observations  widi  amodified  curriculum  that  is  mimitely  adjusted.  Instruction,  thnefore,  is  very 
much  dqiendem  on  indivimial  response  histories.  But  an  alternative  approach  involves  assessirig 
incoming  knowledge  and  skills,  ddier  instead  of,  or  in  addition  to,  emerging  knowledge  and 
skills.  This  alternative  enables  die  curriculum  to  adrqtt  to  both  persistem  and/br  momentary 
performance  information  as  wdl  as  dieir  inteiactimi  (see  Smte,  1993-a,  1993-b).  hi  fact,  many 
have  argued  that  incomir^  knowledge  is  die  single  most  irnportam  determinam  of  subsequem 
learning  (e.g.,  Alexander  &  Jwty,  1988;  Dodiy,  19^  Glaser,  1984). 

Odier  kinds  of  systems  may  not  even  have  a  tutor/coadi  present.  For  examfde,  the 
strength  of  mictoworids  (exidoratory  envircmmems)  resides  in  the  underlying  simulation  and 
explicit  intmfaces  in  tdiich  students  can  freely  cmiduct  experimeids  and  (fotain  results  quickly 
and  safely.  This  is  apatticularly  attractive  feanue  for  domains  diat  are  hazardous,  or  do  not 
feequemly  occur  in  die  real  world.  PUrdiermore,  these  systems  can  be  intrinsically  motivatir^,  in 
terms  of  generating  interesting  cmniriexities  that  keqi  students  interested  in  continuing  to 
exfdore,  while  givhig  diem  sufBdem  succem  to  prevatt  fhistratioa 


•me  T"  in  ITS 

Our  wmldiig  definition  of  computer-tutor  intelligence  is  that  the  system  must  behave 
inteUigently,  not  achudly  be  intelligent,  like  a  human.  More  specifically,  we  believe  that  an 
intelligent  system  must  be  aide  to  (a)  accurately  diagnose  students'  knowledge  structures,  skills, 
andA>r  styles  using  principles,  rather  dian  preprogrammed  responses,  to  decide  what  to  do  next, 
and  then  (b)  adqr  instruction  accordingly  (e.g.,  Clancey,  1986;  Shute,  1992;  Skeman  &  Brown, 
1982).  Moreover,  the  traditional  intelligent  tutoring  system  "...  takes  a  longitudinal,  rather  than 
cross-sectional,  perspective,  focusing  mi  die  fluctuating  cognitive  needs  of  a  single  learner  over 
time,  rafiier  dun  mi  staUe  inter-individual  differences."  (Ohlsson,  1986,  pp.  293-294). 

In  order  to  obtain  a  rough  idea  of  the  degree  of  consensus  among  researchers  in  die  ITS 
community,  twenty  experts  were  asked  to  summarize,  in  aootqde  of  sentences,  their  ideas  mi 
what  the  "I"  in  ITS  meant  Following  are  the  different  re^nses  received  (in  alphabmical  order, 
and  sUgfady  edited,  for  readaMlity). 

Ton  deJong  (Dec.  10, 1993):  InteUigmd  in  ITS  stands  for  the  ability  to  use  (ina 
connected  way)  different  levels  of  abstraction  in  the  representation  of  die  learner,  the  domain, 
and  the  instniction.  The  higher  die  mge  of  distraction,  the  hi^im  the  intelligenoe.  The  phrase 
"in  a  connected  way"  imidies  diat  one  diould  be  dde  to  go  from  qiecific  (e,g.,  log  files)  to 
abstract  (e.g.,  learner  characteristics),  as  well  as  the  odier  way  around  (e.g.,  from  generd 
instructional  strat^es  to  a  qiecific  instrucdonal  transaction). 

Sharon  Derry  (Oct.  15, 1993):  An  intelligeitt  instructional  system  can  observe  what  the 
student  is  doing  during  problem  striving  and/br  has  done  over  a  smies  of  protrimn-solving 
sesaons,  and  fiom  dus  informatimi  draw  infermoes  about  die  student's  knowledge,  beliefs,  and 
atdtudes  in  terms  of  some  theory  of  cogndimi.  Asystemcanbeintelligemudiedierm’notit 
makes  instructional  dedsimis  based  on  this  informadon,  but  if  it  doern'r  use  such  infrnmadan  in 
instrucdonal  dedsimi-making,  dien  I  don’t  diiirir  of  it  as  a  tutoring  system,  but  radier  a  tool  diat 
has  some  diagnostic  cqiabilides. 

Wayne  Gray  (Nov.  15,1993):  I  concede  a  wide  ladtude  on  the  qiplicadmi  of  die  term 
TTS"  in  r^ard  to  instnicdonal  systems.  However,  at  some  level  and  to  some  degree,  diere 
should  be  some  sort  of  "cognitive  modeling"  technology  involved.  The  modeling  can  be  of  an 
ideal  student,  instructor,  or  grader,  or  of  a  less-dian-ideal  problem  solver  as  in  die  "student 
modds"  that  are  often  built  up  in  ITS.  To  be  intelligent,  a  system  has  to  incorporate  and  use  a 
modd  for  making  decisions  about  Miat  to  do  at  any  given  point  during  learning. 

Lee  Gugerty  (Oct.  20, 1993):  Intelligmit  tutoring  involves:  (a)  exfriidt  modelirig  of 
expert  represerriations  and  cognitive  processes;  (b)  detection  of  student  errors;  (c)  diagnods  of 
studeitis'  knowledge  (correct,  incorrect,  and  missing);  (d)  instruction  adapted  to  students' 


knowledge  sUke  (vie  problem  sdecdon,  Mms.  feecttMck.  and  e;qdicit  didactic  instniction);  wd  (e) 
dcdng  all  of  die  above  in  a  timely  fsriiion  as  the  Audent  solves  protdems  (not  post  hoc). 

PatKyUonen  {Oct.  14. 1993):  An  "intelligent"  tutoring  system  is  one  tbat  uses  AI 
programming  techniques  or  principles.  However,  what  is  omsidered  AI  (as  opposed  to  standard) 
programming  dianges  overtime  (e.g..  expert  systems  used  to  be  aidietypal  AI  systems,  but  are 
now  found  in  $1(X)  PC  software  packages).  For  me.  two  features  separate  ITS  software  from 
conventional  CAI.  One  is  foe  existence  of  a  student  model.  What  foe  student  knows  carawt  be 
recorded  directly,  but  must  be  inferred  by  foe  system,  based  on  a  pattern  of  successes  or  failures 
by  foe  student  taid  an  "understanding"  of  what  knowlet^  problems  in  the  curriculum  call  iqxm. 
Anofoer  feature  is  foe  existence  of  "coaches,"  "demons"  or  "bug  libraries"  that  can  observe  a 
student's  behavior  and  either  diagnose  the  behavior  in  terms  of  the  student's  current  knowledge 
stnicture,  or  suggest  corrections  to  that  behavior. 

Susanne  Lajoie  (Oct.  18. 1993):  Hie  "T  in  ITS  means  that  the  computer  can  provide 
additive  ftmns  of  feedback  to  ttie  leamer  based  on  a  dynamic  assessment  of  the  studmifs  "model" 
ofpetftnmance.  Infelligeiitfeedbadt  means  that  die  assessment  of  the  learner  is  ongoing,  the 
feedback  is  iqjpropriate  to  tint  particular  teamer  in  the  context  of  uriiere  an  impasse  has  been 
encountered,  and  it  is  not  canned  but  generated  on  tin  qx)t.  based  on  sludett  needs. 

Alan  Lesgold(Oct.  21, 1993):  "Imelligent"  means  tint  tin  system  uses  inference 
medumisms  to  provide  coaching,  mqjlanation,  or  otiier  information  to  tin  student  performing  a 
tadL  Ptntiier.  it  in^dies  that  tills  inftmnation  is  tuned  to  tbecmtext  of  the  student's  ongoing 
wmk  andAm'  a  modd  of  tin  student's  evtdving  knowledge. 

Matt  Lewis  (Oct.  28, 1993):  An  "intelligeiit"  tutoring  system  contains,  at  a  minimum,  a 
reasonably  general  simulation  of  human  proUem  srdving  indirect  service  of  communicating 
knowledge  and,  like  a  good  human  tutor,  separates  domain  knowledge  ftom  pedagogical 
knowledge.  The  simulation  might  solve  domain-specific  {uoblems  in  the  target  instructional 
domain  (e.g.,  a  human-like  qiproach  and  sohitkm  to  tiie  problem  of  writing  a  fugue)  or  solve 
pedagogical  proUems  (e.g.,  error  diagnosis  and  attribution,  or  sdection  of  qipn^ate  response). 

Wes  Regian  (Oa.  14. 1993):  An  ITS  diffins  frmn  CAI  in  that:  (a)  instructional 
interactimis  are  individually  tuned  at  run-time  to  be  as  efficient  as  possible,  (b)  instnictimi  is 
based  on  cognitive  princiities,  and  (c)  at  least  some  of  tiie  feedbadt  is  generated  at  run-time, 
ratiier  than  being  all  canned.  It  is  not  paiticulariy  important  to  me  what  language  tiie  system  is 
written  in,  whether  or  not  the  system  is  in  any  sense  arguaUy  aware  of  anything,  and  whether  its 
decisions  are  rendered  in  a  manner  that  is  the  same  as  abuman  decision. 

Frank  Ritter  (Oct.  15. 1993):  The  "I"  in  ITS  usutdly  indicates  tiiat  a  single  knowledge- 
based  component  has  been  added  that  helps  a  tutoring  system  perform  one  aspect  of  its 
peiftiimanoeinabetierway.  This  can  be  in  lessra  scheduling,  providing  examines  of  domain 
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knowled^  in  action,  or  providing  domain  knowledge  for  compariscm  with  a  student's  bdiavior. 
Whatitgboyldmeanisdiatitdoesthe  whole  job  intelligently.  The  systems  are  usually  not 
systems  in  the  fiill  sense  of  ttie  word,  they  tend  to  be  prototypes,  with  ti^le  pans  missing. 

Derek  Sleeman  (Nov.  22, 1993):  "InteUigent"  tutming  systems  need  to  have  motivating 
teaming  environments,  to  communicate  effectively,  and  to  render  dynamic  decisions  about 
appropriate  control  strategies.  Since  the  1960s,  we've  seen  that  thesame  material  ddivered  on 
various  systems  differentially  invoke  motivatioii:  thus  we  need  to  confirm  die  factors  that  impact 
a  teainei*s  motivation.  Next,  communication  can  only  occur  itriien  diere's  a  shared  world-view. 

In  conventional  dialogs,  humans  dynamically  tailor  their  language  to  the  person  to  whom  they 
areqieakiitg,botcooq)uter8arenotyet8oadq)tal3te.  Finally,  control  imfdieswhidi  of  the 
partners  in  die  diah%  win  take  the  irddadve,  and  if s  often  necessary  to  dumge  control  during  an 
inmaction,  depewfing  on  die  social  setting,  the  studem's  motivation,  and  the  level  of  incoming 
knowtedge. 

Eittot  Soloway  (Oct.  28. 1993):  The  intent  of  the  1"  in  ITS  was  to  expliddy  recognize 
dMt  a  tmoring  system  needs  to  be  exceedingly  flexibte  in  wder  to  reqiond  to  the  immense  varied 
of  learner  responses.  CAI,  as  die  forerunner  of  ITS,  didn't  have  the  range  of  interactivity  needed 
forteaming.  In  fact,  the  movement  from  ICAI  to  ITS  was  to  further  distanoe  the  new  type  of 
temdng  environments  from  the  rigidity  of  CAL 

Sig  Tobias  (Oct.  15, 1993):  Imenigent.  in  an  ITS  context,  means  diat  the  program  is 
flexible  in  die  mediod  and  sequence  widi  which  instructional  materials  are  presented  to  the 
stndenL  Putthemiore.  die  system  is  capaMe  of  arhfXing  instructional  parameters  to  student 
characteristics  by  using  data  otdlected  prim  to,  or  (taring,  instruction  for  such  decisions.  Hnally, 
it  si^gests  dnt  die  iiatnictional  system  can  advise  die  student  regarding  options  most  likdy  to  be 
successful  for  the  student 

Kurt  VauLehn  (Oct.  18, 1993):  "foteDigent"  means  that  at  least  one  of  the  ditee  classic 
mtxtates  is  iiKduded  indie  tutoring  systmn.  That  is,  die  machine  has  either  a  subject-matter 
expert,  a  diagnostician/taodatt  modeler,  or  n  expert  teacher.  Just  as  in  any  AI  system,  an  eiqiett 
system  widi  oidy  10  proctacdon  rules  is  inteUigem  only  in  that  it  holds  the  possiMlities  for 
mqiaiision;  a  lOO-rute  system  is  moderately  intelligent;  and  lOOOf  rules  means  you're  really 
getting  there. 

Beverly  Woo(f(Oct.  25. 1993).  My  view  of  tutor  intelligence  includes  the  following 
demenis:  (a)  mechanisms  that  model  die  thirddng  processes  of  domain  experts,  tutors,  and 
students;  (b)  environments  dua  supply  world-class  laboratories  within  Mii(h  students  can  buUd 
and  test  their  own  reality;  and  (c)  a  computer  partner  dud  fodlitates  die  ah-ha  experience, 
recognizes  the  student's  imendon,  and  aids  and  advises  die  studott.  An  intelligent  enviroranent 
would  also  siqipoit  oomfdex  discoveries. 
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As  seen  in  this  non-nntlom  sample  of  re^xmses  riMut  Gtmsdtuies  in^ 

ITS,  jost  about  eveiyone  agrees  that  the  most  critical  elemem  is  real-time  cognitive  diagnosis  (or 
student  modeling).  Ihe  next  most  frequentty  died  feature  is  odoiprive  rentttharion.  And  while 
sonre  maintain  feat  remediatian  actually  comprises  die  T*  in  intdligent  tutoring  systems,  our 
position  is  that  the  two  components  (diagnosis  and  remediation),  woridng  in  ooncot,  make  up 
theinielligenoeinanITS(seeourw(nkingdeflidtion,ahove).  Consider  the  case  idiere  a  system 
diagnoses  a  student's  skill  levd,  but  makes  no  efibft  to  rectify  any  faulty  behaviors.  Can  that 
system  really  be  classified  as  intelligent?  Themetically.  peihaps,  but  practically,  no.  Other 
duuacteristics  of  intelligence  qipear  less  frequendy  in  these  reqxmses  (e.g..  canned  vs.  geoeraled 
problems  and  feedback,  degree  of  feamo'oontnd  fan  the  environment,  presence  of  awareness). 

Ihe  d^ree  of  agreement  among  reqxndere  was  actually  surprising  given  the  diverrity  of 
respective  research  interests  and  backgromds  (computer  scientists,  psycfadogists,  educatms). 

But  dds  degree  of  consensus  was  not  dways  there.  Until  fairiy  recently,  die  field  was  not  only 
esoteric,  but  quite  fractionated;  no  two  people  could  agree  on  what  "imelligence"  in  a  computer 
tmorachudlyrefenedto.  To  understand  the  cuireatooQgnienoe.  we  need  to  briefly  jump  back  in 
time  to  see  the  evcdution  of  intelligent  tutoring  systeott,  from  die  late  19ti0s  to  the  present  (mid- 
1990b). 


THE  204-  YEAR  mSTTORY  OF  ITS 
histead  of  discussing  individual  tutoring  systems  that  qiamied  dds  period,  we  present 
salieat  dtaiacteristics  of  systems  appearing  at  various  points  in  time,  iUnstiating  with  exemidar 
tutors.  For  excellent  discussions  of  individud  intelligent  tutoring  systems,  see  the  fidlowing 
books;  Bieiman,  Bieidcer,  Sandbeig.  1989;  Goodyear,  1991;  Lajtde  &  Deny,  1993;  Lawler  ft 
Yazdani,  1987;  Nidtoson  and  2SodhidBS,  1988;  Fdson  ft  Richardson,  1988;  Psotka,  Massey, 
Mutter,  1988;  Regian  ft  Sbute,  1992;  Sdf,  1S>88;  Sleeman  ft  Brown,  1982,  and  Wenger,  1987. 
The  issues,  by  decade,  tint  will  be  discussed  can  be  seen  in  die  following  table. 
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Issues  Related  to  ITS  Develi 


Problem  Ocnenaion 


Siiiq>le  Student  Moddii^ 


Knowledge  Rqnescatation 


Sociaiic'nitating 


Soils  ft  Stiaiegic  Knowing 


Reacdve  Learning  Envinnnients 


BnggyLHnny 


Expert  Sjfsiean  ft ‘nuon 


Overiay  ModelsKIenetic  Qiapli 


Model*Tncing 


More  Buggy'Baaed  Systems 


Case-Based  Reasoning 


Discovery  Worlds 


Itogressian  of  Mental  Models 


SinmlatioiiS 


Natmal  Language  Processing 


Anthoring  Systems 


1990s 


Learner  Control 


Individual  vs.  ColUbontive 
Learning 


Situated  Learning  vs.  Information 
Processing 


>^rtual  Reality 


Hanlware  and  software  have  evolved  at  an  astounding  nie  over  the  past  20  yeais.  Toput 
things  in  per^iective.  consider  the  1970s--*l\ing"  wasthefage(i.e.,asifliij|OeMadi;-and-white 
oomptnerized  tdble  tqmis  game)  and  8K  random  access  memory  (RAM)  die  norm  for  a  PC 
Computer-admirdstered  instnictkin  developed  before  the  1970b  was  inflexible  and  didactic 
because  die  systems  had  very  limited  capabOitxa  (ix.,  memory  capacity  and  computational 
speed)  ftw  adaptive  (fiagnosis  and  feedback.  Fartfaetmote,  "...dm  only  theory  availabie  to  guide 
inatiuctiand  devdopment  was  bdiavkx’ theory,  whidi  pomly  matdmd  die  oogidtive  goals  of 
edncalioa”9je8g(dd,  1988,p.iii).  Over  time,  lesearchers  in  Aland  cogidtivepsydulogy  joined 
finces,  and  togedier  provided  a  basis  for  a  new  genaalion  of  computer-based  teaching  programs. 
Some  of  the  research  issues  diat  dominated  die  70's  are  (Hscossed  bdow. 

p>jii.ritn»  PtnMgm  fianaatioiL  The  earliest  systeoBs  to  incorpmate  some  now  "dasdc” 
ITS  etanents  were  programs  diat  generated  problems  and  teaming  tadffi.  representing  a  big 
dqiarture  from  the  canned  problems  stored  in  CAl  databases.  For  example,  Ufar  (1969) 
developed  a  conqmter-based  learning  system  that  created,  in  teal-dme,  simple  arithmetic 
pnibienis  and  vocabulary  recall  tasks.  The  next  major  advance  in  this  area  came  in  the  fimn  of 
onmpmBrpmgiamxthm  generated  problems  tfiat  had  been  tailored  to  dm  knoudedge  and  skill 
level  of  a  particular  stndoit,  thus  providhig  the  foundation  for  student  modeling. 

simpte  .ntiMtent  MmteHny  The  Baslc  Instructional  Ptogrun  (BIF)  develops  procedural 
iftiiia  required  in  teamhig  die  programming  language  BASIC  (Barr,  Beard,  and  Addnson,  1976). 
It  did  80  tw  sdecdr^  pioUems  based  on  what  the  student  already  knew  (past  performance). 


vMdi  skills  should  be  tau^  next,  and  its  analysis  of  the  skills  required  (proUems  in  the 
curriculum).  Exercises  were  dynamically  and  individually  selected  per  person  (fnnn  a  pool  of 
100  sample  problems);  then  teaching  heuristics  were  apidied  to  the  student  model  to  identify 
skills  to  be  taught  and  exercises  were  selected  diat  best  involved  diose  skills.  Sdectimiof 
Htpropiiate  exercises  was  based  on  informmion  ooittained  in  a  network  called  the  Curriculum 
Informrttion  Netwmk  (ON),  rdatirtg  tasks  in  the  curriculum  to  issues  in  the  domain  knowledge. 
Thus,  a  programming  task  in  the  tutor  was  represented  in  terms  of  its  component  skill 
requirements.  Based  on  a  task  arudysis,BlP  knew  that  die  comporrertt  skills  needed  for  solving  a 
particular  programmiiig  pnddem  included  such  skills  as:  initialize  numeric  variaUe,  use  for-next 
loop  with  literal  as  final  value,  and  so  forth.  Moreover,  eadi  task  tapped  a  number  of  ddlls. 

Rf  pwwwtfatinn.  dassic  CAI  used  pages  of  text  to  represent  knowledge,  but 
witii  little  psychological  validity.  In  contrast,  CarboneU's  (1970)  SCHCX^AR  program  (often 
crediled  with  being  the  first  true  ITS)  used  a  semantic  net  to  represent  domain  knowledge  (Soudi 
American  geography)  as  well  as  the  studrat  model  Nodes  in  the  network  had  tags  to  indicate 
udiedier  the  concept  was  known  to  the  student  lliis  novel  apfdicatimi  of  semantic  network  as  a 
general  structure  of  knowle^  rquesentatim  siqiporied  mixed-initiative  dialogs  with  studems. 
Not  only  could  die  computer  ask  ^lestions  of  die  student  but  die  student  could,  theoretically,  ask 
questions  of  die  corrqwter.  One  major  limitatioi  of  this  semantic  knowledge  representation  was 
the  difficulty  of  rqxesenting  procedural  knowledge. 

Socntic  Ttonriny.  Caibonell's  research  ^Mwned  another  line  of  work  ccMioenied  with 
enabling  systems  to  engage  in  Socntic  dialogs,  believed  to  involve  the  learner  mote  actively  in 
die  learning  process.  Collins  (1977)  outlined  a  set  of  tutorial  rules  for  Socratic  tutoring  that  were 
incmpocated  into  a  system  called  WHY  (Stevens  and  Collins,  1977).  For  example,  con^der  the 
ftdlowing  IF/THEN  string:  IF  the  student  gives  an  exfdanation  of  one  or  more  factors  that  are  not 
sofiBdem,  IHEN  formulate  a  general  rule  ftir  asserting  that  the  given  fisetors  are  suffidmit,  and 
ask  the  student  if  die  role  is  true  (Collins,  1977,  pp.  343-344).  Instead  of  semantic  nets,  the 
domain  knowledge  (rainfall)  was  stored  in  a  "scr^khieiarchy"  containing  infotmatiem  about 
stereo^pical  sequences  of  events. 

sariiiK  and  Snateyic  Knnwieity-  AnotiKT  attempt  to  Stimulate  thought  among  Students 
(radier  dian  being  passive  lec^ents  of  infimnation)  was  the  ftrcus  of  a  group  of  researchers  at 
PARC  in  the  mid-  to  late-1970s.  For  instance,  WEST  (Burton  &  Brown,  1976)  was 
devdoped  to  he^  studems  learn^ptactice  skills  involved  in  the  manipolation  of  aridunetic 
mqxessians.  'niegoalwastomoveatoond8gameboard(ff(nvrke  West  Was  Won)  and  either 
rvlvimoe  the  maximum  number  of  squares,  land  on  and  dnis  Iximp"  an  opponent  badt  some 
fixed  ranmirtt  of  squares,  or  take  a  shortcut  Not  tmly  was  basic  arithmetic  skill  involved,  but 
aifn  stiat^c  knowtedge  was  retpiiied.  Hk  system  was  attentive  to  all  levels  of  knowledge  and 
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ddn,  but  tbe  "coach"  was  somewhat  unobtiusive,  sitting  in  the  background  monitoring  the 
student's  moves*  intervening  only  when  it  was  dear  that  the  student  was  floundering.  Thenthe 
coadi  would  make  a  few  suggestlmis  to  enhance  studem  skills.  WEST’S  coaching  goals  were 
acccmidisiied  by  focusiiig  on  the  strategy  used  to  constnict  a  move  (viz..”issue-based"  tutoring). 

Pi»«rriv>!  I  Rnvimnmente.  In  reactive  learning  environments,  the  system 

responds  to  teamers'  actions  in  a  variety  of  ways  that  extend  understanding  and  help  dumge 
entraidiedbdief  structures  using  examines  fliat  challenge  the  learner's  currem  hypotheses.  An 
early,  excellent  example  of  this  kind  of  environment  was  SCH^HIE  (Sophisticated  lnsAnK:tional 
Environment),  designed  to  assist  learners  in  devdqring  electrmic  trouUeshooting  skills  (see 
Brown  &  Burton,  197S;  Brown,  Burton,  &  deKleer,  1982).  For  instance,  in  S(X*HIE  I,  learners 
located  faults  in  a  broken  piece  of  equipment  lliey  could  adc  SCVHIE  questions  in  English 
(e.g.,  to  obtain  values  of  various  measurements  takmi  on  the  device).  SCX^IE  I  induded  three 
main  componerns:  a  mathematical  simulatioii,  a  inogram  to  understand  a  subset  of  natural 
language,  and  routines  to  set  up  contexts,  keq>  history  lists,  and  so  (ML  Astudent 
troifelediooting  a  simulaled  piece  of  ecpiipment  could  offer  a  hypothesis  about  what  was  wrong. 
St^PfflE  I  reacted  to  the  request  by  comparing  the  hypothesis  to  the  measumnente  entered  by  the 
smdent.  S(X*HIBII  extended  tire  environment  of  its  predecessor  by  adding  an  articulate  erqrert 
based  on  a  pie-stoted  dedsian  tree  fw  troubleshooting  the  power  supply  that  was  annotated  with 
schema  for  pnxhicitigexplaiiatians.  SOPHIE  HI  represented  a  significant  advance;  it  contained 
an  underlying  expert  based  on  a  causal  modd  rather  than  on  a  mathematical  simulatioiL  The 
inqMMtance  of  fliis  change  is  ttiat,  in  SCKWE 1,  fire  simulator  worked  out  a  set  of  equations  not 
using  human-like,  causal  reasoning,  so  it  wasn't  possiUe  for  die  system  to  exfdain  its  decision  in 
any  detafl.  But  SCXWE  m  did  emidoy  a  causal  irKxlel  of  circuits  to  deal  with  die  student 
feedback  defteiency.  Research  widiSCXWE^wned  a  lot  of  later  research  in  troubleshooting, 
reactive  teaming  environmerris,  and  articulate  eiqrerls. 

Buggy  Lihcarv.  Brown  and  Button  (1978)  also  developed  BUGGY,  a  fteciuerdly  cited 
CTMipte  of  a  system  emidoying  a  "boggy"  liluaty  ^rproach  to  the  diagnosis  of  student  errors. 
BUGGY  was  a  fitamework  for  modding  misc(Micepti(Mis  underlying  procedural  errors  in  addition 
and  sdNraction  where  students'  errors  were  represented  as  the  results  of  "bugs"  (errors)  in  an 
odietwise  ccuiect  set  of  procedures.  KBUGGY  (Buttcrn,  1982)  was  developed  as  an  off-line 
version  of  die  system  based  on  die  BUGGY  framework  using  the  pattern  of  errors  fircMn  a  set  of 
proUons  ttxronsttuct  anl^podiesis  amoeming  a  bog,  oroHnbinafkHi  of  bugs,  from  die  library 
dwtgmeiaieddieerrcns.  Il^UGGY  (Burton,  1982)  was  an  on-line  verskMi  of  BUGGY, 
die  stndem's  procedure  tdt-by-Mt  while  giving  die  learner  new  proUems  to  solve. 

The  «naj(w  limitatton  of  these  kinds  of  systems  was  the  inaldlity  to  anticipate  all  possible 
mteconcqitioiis.  Moreover,  bugs  could  appear  transioit  as  they  were  being  repaired. 


R¥p(»ff  Svsfiims  «nri  Tutors  MYCIN  (Siortliile.  1976)  was  a  rule-based  expert  systm 
for  diagnosiiig  certain  infectious  diseases  sudi  as  meningitis.  GUIDON  (Clancey,  1979)  was 
constructed  to  interface  wife  KfYQN  for  tutoring,  interactively  presenting  the  rules  in  foe 
knowledge  base  to  a  student  This  tutoring  qpeiated  as  follows.  GUIDON  described  case 
dialogs  of  a  sick  patient  to  foe  studmtt  in  general  terms.  The  student  had  to  adopt  foe  role  of  a 
physician  and  ask  for  information  foat  might  be  relevant  to  the  case.  GUIDON  cranpared  foe 
student's  questhms  to  foose  triiich  MYCIN  would  have  asked  and  then  reqxmded  accordingly. 

Qveriav  Modds/Genedc  Graph  The  defitiitioa  of  an  overlay  model  is  one  of  a  novice- 
apeit  difference  model  rqnesenting  missing  concqrtions.  If  s  tyfucally  implemented  as  either  an 
erqrert  model  annotated  for  missing  items,  or  an  expert  model  with  weights  assigned  to  each 
dement  in  foe  expert  knowledge  base.  To  illustrate  how  it  works,  craisider  WUSCXl  (Stansfidd, 
Carr,  &  Goldstein,  1976)-foe  name  of  foe  on-line  coach  for  the  game  WUMPUS  (Yob,  1975). 
Ihe  WUMPUS  fdayer  had  to  traverse  forough  successive  caves  to  locate  the  hiding  WunqMS. 
Many  dangers  faced  foe  (dayer  (e.g.,  pits,  bats),  but  foe  ptoUem  could  be  solved  by  apidying 
logicd  and  probaldlistic  reasoning  to  infonnation  obtained  along  foe  way.  The  goal  of  foe  game 
was  to  shoot  an  arrow  into  foe  Wurnpus' hiding  spot  behne  you  were  killed.  WUSORevolved 
through  (at  least)  three  genetadoiis,  each  wifo  a  progressively  more  sophisdcaied  student  modd. 
The  first  vetdm  had  oidy  an  erqiert  aod  advisor  and  did  not  try  to  diagnose  foe  learner's  state  of 
knowler^.  Ihe  next  version  (11)  incorporated  an  overlay  model  (Carr  A  Goldstein,  1977)  where 
the  erqrertise  was  reinesented  as  rales,  rmd  foe  studenf  s  knowledge  state  was  a  subset  of  the 
eaqteifs  knowledge.  Gtddstein  (1979)  made  foe  final  transfinmation  to  WUSCXl  (HI)  by 
including  the  genetic  graph,  combinirig  overlay  modding  (rule-based  representation)  wifo  a 
leatner-orientedsd  of  litdts  between  curricular  elements.  'Xjenetic'' related  to  die  notitm  of 
kttowledge  bdng  evolutionaty,  ard  graph  denoted  fire  telafiraiships  between  parts  of  knowledge 
erqrressed  as  links  in  a  network.  A  genetic  grqfo  could  reptesem  ^pe-of-links  (e.g., 
generalization,  andogy,  refitKment)  as  well  as  deviation  links  (Le.,  buggy  rules  as  t^rposed  to 
sitrqdy  disent  ones). 

The  1970s  were  marked  by  erqietimental  systems  that  bore  little  resemUanoe  to  (Hie 
tmofiier.  Dorir^  the  following  decade,  systems  became  less  idiosyncratic,  but  ttiete  was  dill  a  lot 
of  diversity  in  foe  fidd 

.<8t«iri«rrliy.ed  Anpmadies  and  Envimnments 

The  1980s  were  characterized  by  enormous  growth  and  m(Hnentum  in  foe  ITS  field  By 
the  mid-llWOs,  die  develc^mient  of  tutors  greatly  exceeded  their  evaluations;  everyone  wanted  to 
participate  in  die  excitmnent  of  building  ITS,  but  few  cared  to  test  their  system's  efficacy  (Baker, 
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1990;  Littmai  A  S<doway,  1988).  Sleeman  (1984)  attempted  to  ibcos  leseaidi  efforts  by 
ootUniiig  four  main  proUems  with  ITS  at  the  time: 


(1)  Feedback  ipecj/Icity-Instittctional  feedback  was  often  not  sufficiemly  detailed  for  a 

particular  learner. 

(2)  Afon-odoipiiaM/iry-Systems  forced  students  iitto  their  own  ooncqjtual  Cramewortc  rather 

than  adapting  to  a  particular  student's  conceptualization. 

(3)  AdteoreUcal  foundation~T\dmiDg  strategies  used  by  the  systems  lacked  a  theoretical 

cognitive  foundation. 

(4)  Restrictive  enviroament-Vser  interactimi  tnd  e:qdorati(m  was  too  restricted. 


These  main  criticisms  were  addressed,  to  varying  degrees,  during  ttie  1980s. 

Mndei  iraciny.  AndeiscMi  and  his  colleagues  at  Cam^e-hfeUon  University  developed  a 
model-tracing  i^tproadi  to  tutoring  based  rm  production  systems  as  a  way  of  modeling  stodent 
behavior.  Hie  model-tracing  iqipioadi  has  been  emi^oyed  in  a  variety  of  tutoring  systems,  sudi 
as  the  LISP  tutor  (Anderson,  Boyle  ft  Reiser,  1985)  and  ftie  Geometry  tutor  (Anderson,  Boyle  ft 
Yost,  1985).  Model  tracing  provides  a  powerful  way  to  both  validate  cognitive  ttieories(e.g., 
Anderson,  1987)  and  to  ddiver  low-level,  personalized  remediatioiL  The  approadi  works  by 
delineating  many  hundreds  of  production  rules  that  model  curricular  "dninks”  of  cognitive  skill. 
A  learner's  acquisition  of  these  chunks  is  monitcned  (U.,  the  student  model  is  traced),  and 
dqNature  fiom  die  optimal  route  is  immedialely  remediated. 

b  theory  (ad  practice)  the  model-tracing  reproach  for  die  Geometry  and  LISP  tutms  is 
so  comidetediat  it  captures  a  erwnnouspercenh^  of  ad  students’ errors.  A  major  drawback  is 
dutddsapproadi  does  not  allow  students  to  commit  ttwse  errors  diemselves.  As  soon  as  there  is 
a  tnis-stq>,  die  tutor  cries  "foul"  and  stops  the  student  from  doing  aything  else  until  die  correct 
stqi  is  taken.  As  Reiser  points  out  (e.g..  Reiser,  Ramey,  Lovett  ft  Kimberg,  1989),  the  studot 
is  not  only  prevented  from  following  these  mistakes  to  their  logical  condusirm  (and  gettnig 
hopdesdy  confused)  but  also  prevented  from  obtaining  a  insight  into  the  mistake  (i.e.,  that  the 
mistake  is  obvious).  Hiese  are  smne  of  the  best  learning  experiences  students  ca  have,  but  diey 
appear  to  be  Mocked  1^  the  model-tracir^  approach. 

Model  tracing  diallenges  the  first  criticism  (feedbadt  speciffcity).  That  is,  the  grain-size 
of  feerAMdt  is  as  small  as  you  ca  gd  (i.e.,  die  production  levd)  thus  providing  the  most 
detailed,  specific  feecRiack  possible.  However,  in  smne  cases  (i.e.,  for  certain  students  or 
particular  proUenas),  this  levd  of  feedback  may  be  too  demental,  the  forest  is  lost  for  the  trees. 
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Next,  as  oaentioiied  above,  the  systons  can  adapt  to  a  wide  range  of  student  conceptualizations, 
challenging  the  second  (non-adjq>tability)  criticism.  The  q)piDach  also  demolishes  die  third 
crdidsm  (atheoretical  foundation),  as  it  was  exididdy  based  on  Anderson's  cognitive  theory 
(ACT*).  Hk  positive  features  of  liiis  approach,  however,  are  achieved  at  the  expense  of  the 
fourth  (restrictive  environment)  criticism.  That  is,  the  model-tracing  approach  is  restrictive.  To 
accom[dish  die  necessary  low-levd  OKHiitoring  and  remediation  of  this  approach,  the  learner's 
fieeckxn  has  to  be  curtailed.  So,  learning  by  one's  mistakes  is  out  (which  is  often  a  powerful  way 
to  learn).  A  final  drawback  of  this  r^iproach  is  that,  while  it  works  very  well  in  modeling 
procedural  skill  acquisition,  it  does  not  work  well  for  domains  that  are  ill-structured,  or  that  are 
not  rule-based  (e.g..  Creative  writing,  Economics,  Russian  history). 

More  Buggv-ha.sed  Systems.  During  this  time  period,  a  {dethora  of  tutors  was  developed 
based  <hi  die  Ixiggy"  IftMary  qiproadi  (see  BUGGY,  above).  WhUe  these  systems  do  provide 
very  spedfic  feedbadc  about  die  nature  of  the  learner’s  error  (countering  criticism  1,  feedback 
qiecifidty),  the  system  leqxHise  is  dependent  (m  the  program’s  ability  to  match  die  studem's 
error  with  that  of  a  stmed  Txig."  Along  these  same  lines,  as  with  model  tracing  (because  only 
simed  bugs  are  acknowledged),  novel  bugs  are  ignored;  thus  there  is  no  way  to  update  the  buggy 
library  or  adapt  to  the  learner's  current  omceptualization  (criticism  2,  non-adqitability).  This 
approach  is  dieoretically  based  cm  the  notkr?  of  active  errors  in  specific  procedures,  impasse 
learning,  and  repair  theory  (VanLdm.  1990),  countering  criticism  3  (adieoretical  foundation). 
Hnally,  these  systems  constrain  the  learner  smnewhat  less  dian  the  model-tradng  rpproadi;  thus, 
it  is  a  re^xmse  to  critidsm  4  (restrictive  environment). 

A  good  illustration  of  a  system  based  <m  the  buggy  approach  is  PROUST  (Jdmson, 

1986;  Littman  &  Soloway,  1988),  designed  to  diagnose  rxmsyntactic  student  errors  in  Pascal 
{XOgrams.  The  system  works  by  locating  errors  in  studmits'  programs  where  they  compute 
varkHis  descriptive  statistics  such  as  the  minimum  and  maximum  values,  and  averages.  The 
major  drawback  of  this  system  is  diat  it  is  imidemented  off-line.  In  other  words,  die  tutor  has 
access  to  a  final  inoduct  rai  adiidi  to  base  its  di^nosis  of  student  errors-comifleted  student 
programs  are  submitted  to  PROUST,  vdiich  prints  cnit  the  diagnosis  (Johnson  &  Soloway,  1984). 

A  parallel  ’huggy"  research  {Roject  involved  a  system  called  PIXIE  (Sleeman,  1987),  an 
m-line  ITS  based  on  die  Leeds  Modeling  System  (LMS),  a  diagnostic  model  for  determining 
sources  of  mror  in  algelna  {Hoblem  solving  due  to  incorrect  procedural  rules  or  "mal-niles." 
While  some  may  equate  mal-niles  widi  buggy  rules,  they  differ  in  a  fiindamoital  way.  Sleeman 
created  diem  by  postulating  a  set  of  basic  buggy  rales  fimn  which  higher  order  mal-rules  could 
be  generated  from  the  fracture  of  the  knowledge  base  itself.  Mal-rules  are  inferred  from  basic 
prindides  and  bugs;  diey  are  at  a  levd  of  abstraction  above  bugs.  In  fact,  John  Aralerson  makes 
the  same  ptdnt  about  his  modd-tracing  fnocedures.  Because  of  die  comidexity  of  his  model- 
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tradiig  productkms,  many  productkms  fire  or  are  used  over  and  over  again  in  contexts  for  which 
they  were  not  first  generated,  and  so  fitey  too  take  on  a  kind  of  abstract  or  general  quality  in  his 
fiamewoik. 

The  major  problem  with  LMS  is  that  it  tmly  diagnoses  the  incorrect  rules;  it  does  not 
ranediate. 

rase-Based  Reasnniny  Another  category  of  systems  emerging  at  this  time  came  frmn 
case-based  reascming  (CBR)  research  (Schank,  1982;  Kolodner,  1988).  Proponents  of  this 
qiproach  suggest  that  the  goal  of  ITS  should  be  to  teach  cases  and  how  to  index  them.  Given 
fiiat  file  student,  not  the  program,  is  the  one  doing  the  indexing,  this  system  affords  the  learner 
greater  freedom,  and  promotes  a  more  adaptive  learning  environment  (countering  critidsms  4- 
lestiictive  envinmment  and  2~non-adi9itaMlity,  respectively).  Fuitheimore,  whereas  the  model- 
tracing  tutors  work  pooily  in  ill-stiucmred  domains,  CBR  woiks  well  in  those  areas  (e.g., 
politics,  philosophy).  Ihis  tradeoff,  however,  can  result  in  less  specific  feedback  to  learners 
(criticism  1,  feedback  specificity). 

These  CBR  systems  also  perform  well  in  domains  where  there  ate  too  many  rules,  or  too 
many  ways  in  n^di  rules  can  be  q)(riied  (e.g.,  programming,  game  {fiaying).  CBR  suggests 
t^projontate  answers  to  oomffiex  proUems,  therdiy  limiting  how  many  rule  cnnlutuuions  should 
beexffiored.  There  are  two  main  processes  invdved  with  CBR:  indexing  (labeling  new 
experimoes  for  future  rdrieval)  and  adaptation  (changing  a  retrieved  case  to  fit  a  curroit 
sitoation).  Ftnfiier,  two  kinds  ofindioes  are  te^riiedrconctme  and  abstract  Concrete  indices 
refer  to  objects  and  actions  usually  directly  mentioned  in  file  case,  while  abstract  indices  refer  to 
more  general  cbaracterizafions.  Ihe  "indexing  protfion”  deals  with  ways  to  deteimine  the  coriea 
abstract  and  ooncrete  incfices  for  cases.  How  one  indexes  new  cases  detennines  what  cases  one 
will  compare  file  iiqruts  against  Using  a  general  index,  one  can  retrieve  a  case  even  when  it 
shares  no  qiecific  details  wifii  the  current  situatitm. 

Srhank  has  made  some  very  provocative  statements  about  die  human  mind  as  a  story 
Idler,  and  about  the  need  to  encq»ulate  knowledge  into  stories,  not  into  hierarchical  data 
structures  like  semantic  networks.  But  his  procedures  have  yet  to  lead  to  any  of  file  other  strong 
diaracteristics  of  ITS  that  we  emphasize  in  this  paper,  student  models,  teaching  models,  bugs, 
and  so  on.  Instead,  they  exist  as  very  generative  and  interesting  systems.  As  such,  they  have 
somefiiii^  in  conmurn  wifii  microworlds;  fiiat  is,  peofde  enjoy  erqilorii^  fiiem  and  can  learn  fimn 
than,  parficularly  those  regarding  ill-structured  and  complex  dranains.  However,  when  students 
don’t  learn,  or  manifest  smne  miscoooeption(s),  the  very  same  looseness  of  structure  and 
organization  in  fiiese  systans  prevents  them  ftmn  daermining  vfiiy ,  and  doing  smnething  about 
it  Rnally,  according  to  Riedieck  and  Schank  (1990),  case-based  reascming  (CBR)  serves  as  a 
model  of  cognition  and  leamiiig.  But,  while  these  systems  present  a  provocative  and  well- 
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conceived  approach  ttiat  has  many  practical  and  obvious  merits,  they  cannot  be  said  to  possess  a 
solid  dieoietical  foundation  (criticism  3,  atheoretical  foundatitm). 

A  major  Umitalion  of  this  approach  includes  the  proUem  of  anticipating  and  representing 
a  sofBdent  number  of  cases  to  be  cataloged. 

Discoverv  Worlds.  With  just  a  few  exceptions,  learning  from  computers  in  the  1960s 
and  1970s  was  diaiacterized  by  inflexible  {Hesentatimis  of  didactic  material.  But  an  t^rposititm 
movement  arose  in  the  1970s  that  gained  steam  in  the  1980s;  it  resulted  in  the  developmem  of 
discovery  learning  environments.  Hiese  computerized  systems  (typically  a  computer  simulation 
envimunent  with  simide  interface  and  tools)  were  designed  to  make  it  possiUe  for  students  to 
acquire  various  knowledge  and  skills  on  ttieir  own.  For  examjde,  students  could  learn  LOGO 
(Papert,  1980)  or  Newton's  laws  of  motion  (White,  1984)  within  discovery  (or  micro)  worlds. 
Typically,  feedbadr  was  "natural"  or  imfdidt,  not  iqrecificaUy  explained  to  the  learner  (relating  to 
critidsm  1.  feedbadt  qiedfidty). 

One  of  die  main  strengths  of  these  systems  was  their  great  adrqjtability  to  a  range  of 
different  learners  (counterir^  criticism  2,  nm-adi^rtatrility).  Students  were  free  to  erqdore  and 
act  within  die  microworld  as  they  chose;  with  the  ramificadons  of  their  actions  immediately 
revealed,  countering  criticism  4  (restrictive  envimiment).  This  movement. was  based  on  die 
tbeoreticai  premise  diat  in  discovery  learning,  one  cm  radically  alter  the  peicqNualrelatitmship 
between  die  learner  and  the  knowledge  or  skills  to  be  acquired,  dnis  addressing  criticism  3 
(adieordical  foundation).  Hus  positimi  was  qntomized  by  Piaget  (1954)  who  staled  dut,  "...m 
ethication  whidi  is  m  active  discovery  of  reality  is  superior  to  one  that  consists  merely  in 
providiiig  the  yoimg  widi  ready-made  wills  to  will  with,  and  ready-made  truths  to  know  with." 

A  major  diawbadr  of  these  systems  is  dut  not  an  persms  are  skilled  in  the  requisite 
inquiry  behaviors  necessary  to  achieve  success  in  these  environments  (see  Shute  &  Glaser,  1990). 
Hut  is,  to  be  mccessfiil,  m  individoal  should  be  able  to:  formulate  efBcient  experiments,  state, 
confirm,  andfor  n^ate  t^podieses;  rppropriately  relate  hypodieses  arrt  experiments;  {dm  future 
experiments  and  tests;  engage  in  self-monitoTing,  and  so  (hl 

Progression  of  Mental  Mndds.  White,  Ftederiksen  and  their  coHeagues  (Fredetiksen. 
White,  Ctdlins,  &  Eggan,  1988;  White  &  Frederiksen,  1987;  White  &  Horowitz,  1987) 
incmporated  ideas  from  (a)  AI  research  cm  mental  models  and  (b)  qualitative  teascming  to 
develop  QUEST  ((^ulitative  Understanding  of  Electrical  System  TrouUeshooting)  as  weU  as 
"Hiiidrer  Tools."  This  qtfHoach,  like  model-tracing,  above,  is  thus  theoretically  grounded  (in 
opposition  to  criticism  3,  athemetical  foundation). 

Hiese  sy^ems  work  by  motivating  studmts  to  want  to  team  by  pointing  out  errors  and 
inconsisimicies  in  their  current  beliefe.  Thm  studeius  are  guided  through  a  series  of 
microwoilds,  eadi  more  ctmiiriex  dun  the  one  i»eceding,  toward  the  objective  of  more  precise 
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mental  models  of  die  evolving  subject  matter  (e.g..  electrical  omicqits  or  Newtcmian  mechanics). 
Hnally,  students  formalize  their  developing  meidal  models  by  evaluating  a  set  of  laws  descriUng 
phenomena  in  die  miciDwoild;  then  they  apfdy  die  selected  law  to  see  bow  well  it  jxedicts  real- 
world  events. 

These  systems  promote  learning,  neither  ounpletely  free  nor  overly  restricted  (relating  to 
criddsm  4,  restricdve  envinmment),  that  reades  about  hallway  between  true  discovery 
environmoits  and  model-tracing  environmertts.  A  (nogrammed  series  of  mental  models  produces 
higher-level  feedback  compared  to,  for  exafflfde,  feecBiack  at  the  production  level  (addressirig 
criddsm  1,  feedback  spedfidty).  Finally,  the  systems  can  adapt  to  a  wide  range  of  learner 
misooiiceptioos  (challenging  criddsm  2,  non-adtiitability). 

SimniirtinnK  Qrtfihical  simuladons  have  beccHne  mote  Central  to  the  ITS  enterprise  as 
die  power  of  computers  has  growtt  Almig  with  inoeasing  computadonal  power,  software 
systems  have  grown  more  oom(dex;  objea-oriented  systems  can  now  mimic  devices  of  great 
complexity  and  interacdvity.  Simulations  ate  useful  nlierever  real  objects  are  involved  in  a 
teaming  or  training  task,  and  diey  provide  many  benefits  over  real  devices.  Not  only  are  they 
less  dangerous,  less  messy,  and  exacdy  rqdicatde;  simulations  are  inspectaUe  and  sdf- 
expianatory  in  ways  diat  real  objects  cannot  be.  Simulatkms  not  only  display  aggregate 
bdiavior,  but  they  are  decomposdde  into  cmsdtuents  that  mimic  novice  or  expert  mental 
models.  This  decomposabUity  of  grqihicdisidays  and  simulatimis  mimics  die  power  of 
pioductions  in  expert  systems  for  creating  natural  dmriks  diat  promote  learning. 

Early  ITS,  like  SCVHIE,  could  generate  tmly  very  simide  line  drawings.  . .  dramadc 
increase  in  die  power  of  grnphic  simulations  todt  place  with  Steamer  (Hdlan,  Hnci  bins  & 
Weitzman,  19S4)nd  die  use  .of  personal  USPmadilnes.  These  maidiines  could  generate 
hiterBcdvegra{diic8widianiittatBd  components.  It  was  not  long  before  (bis  gra^cal  power 
became  avaUaUe  for  ITS  on  smaUer  personal  oraqaiteTS  that  could  be  used  in  industrial  and 
educafional  settinigs.  Of  course,  more  powerful  systems  diat  were  devdtqied  in  the  19805,  like 
Hawk  MACH-m,  could  expand  the  number  of  components  and  complexity  of  die  animadons  tty 
orders  of  magnitude  (Kurland,  Granville,  &  MacLaughlin,  1992).  Using  objea-oriented 
constructions,  MACH-m  made  each  part  of  comi^x  radar  systems  insqiectable  and  sdf- 
exfdanatmy.  For  teaching  trooMesliooting,  each  decomposable  part  of  die  radar  device  could 
even  eiqidain  its  role  in  die  trooUeshootii^  setyience  fin  any  fimlt  that  had  been  created  in  die 
system.  Given  this  power  and  comfdexity,  these  systems  were  stretOied  to  their  limits  and 
brought  to  their  knees  by  additiaial  requirements  for  student  modds,  curriculum  sequences,  and 
hypertext  irtteifeoes.  Even  though  these  conqxitersimulatkMis  were  forced  to  opente  at  die  edge 
of  dieir  acceptability,  an  offidd  Army  evduadoi  verified  the  many  benefits  of  simulation-based 
training  systems  (Farr  and  Psolka,  1992). 
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Depoidlng  on  die  level  that  a  simulated  device  has  been  decomposed  to.  and  the  degree 
of  teamer  reqxmse  regarding  manipulations  and  ensuing  ramificati(ms,  feedback  could  attain 
various  levels  of  detail  (criticism  1,  feedback  qrecifidty).  FUfthennore,  as  simulations  become 
typically  veiy  reactive  to  learner  actions,  they  can  serve  as  a  direa  diallenge  to  the  second 
criticism  (ncm-adqitability).  Simulations,  similar  to  discovery  wotlds.  also  leave  quite  a  Ut  of 
freedcmi  to  exfdore  and  manipulate  simulated  objects  and  devices  (countering  criticism  4. 
restriction  envinmments).  However,  ttre  drawback  of  these  systems  is  that  a  solid  theoretical 
basis  is  ladring  (criticism  3,  adieoreticai  foundaticm).  Simulatim  research  in  the  1980s  qurred 
later  work  tiutt  attempted  to  incoipoiate  pedagogical  strategies  into  the  simulation-based  systems. 
MoreovCT,  related  devdopments  continue  to  evolve  in  cmni^exity  wife  the  additi<m  of  Virtual 
Reality  interfaces  to  three  dimensimial  models  and  simulations  ( Acchione-Noel  and  Psofea. 
1993). 

Two  other  areas  of  researdi  and  develqpment  gained  prominence  at  this  time:  Natural 
language  processing  (NLP)  and  authoring  shells.  While  feese  research  qfeeres  were  importam  in 
relation  to  ITS  research,  feey  could  be  q)|feedwifein  a  variety  of  tutor  types.  For  example,  NLP 
could  be  used  to  communicate  information  to  the  teamer  (or  accqtt  input  flom  fee  learner)  in 
model-tracing  tutors,  discovery  wotlds,  and  so  fcnth.  And  authoring  shellscould  be  built  forthe 
developmemof  a  range  of  tutoring  systems.  Because  of  feis  openness,  fee  following  two  ITS- 
relafed  issues  wont  be  discussed  in  relation  to  our  four  criticisms,  listed  earlier. 

Natural  I  juuniaye  Pmoeadny.  This  technology  was  an  important  part  Of  ITS  right  from 
theb^hming.  SCXWE,  in  feet,  was  built  on  a  powerful  and  original  NLP  technique  developed 
by  Richard  Burton;  it  was  called  Semantic  Grammar.  Representing  a  powerful  combinatiCTi  of 
carefully  selected  keywmds  wife  algmitlnns  feat  searched  fee  context  for  meaningful  variables 
and  objects,  it  worked  surprisingly  wdl,  given  its  relative  simidicity.  Since  communicathm  is 
sudi  an  impmtant  elementof  ITS  (see  Wenger,  1987  fm  emphasis),  it  is  not  surprising  feat  NLP 
tedmidogies  have  been  used  in  several  ITS  for  discourse  networks  (Woolf,  1988)  and  especially 
for  Imgiiagft  Insfnictinn  (Yazdani.  1990;  Paotica,  Hfdland.  &  KersL  1992).  The  development  Of 
powetfoL  efBcioit  Pndog  compitets  and  languages  on  PCs  has  ted  to  the  imidmnentatirm  of 
some  interesting  instructkmal  grontnars  that  can  handle  discourse  in  Englidi  or  ofeer  languages, 
and  provide  multimedia  instruction  in  advanced  language  concepts  and  grammar,  as  well  as 
Kfanpie  vocabulary  and  verb  dectensiotL  The  potential  addition  of  animatkms  and  inunerskm  into 
Virtual  Environmeitts  adds  a  Mght  new  jno^rect  to  fee  (dd  goal  of  feimersfve  language  teaming. 

Aiithnriny  .Sysrems.  The  cTcatimi  of  compufeT-based  environments  to  facilitate  the 
design  and  development  of  ITS  has  been  an  importait  and  continuing  feread  of  research.  The 
goal  of  aufeoring  systems  is  to  give  relative  computer  novices  a  software  toolkit  to  t^ 
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advairtage  of  the  power  of  oomputeis  for  designing  instniction.  An  examfde  of  one  pow^fol 
gnqihic  audwring  system  developed  over  die  last  decade  is  diat  by  Towne  and  Munio  ( 1992). 

Quite  powerful  CBT  systems  have  been  made  availaUe  over  the  years.  Research, 
b^inidnginthe  1980s,  attempted  to  adapt  such  systons  as  authoring  shells  for  develo|dng  ITS. 
Miller  and  Lucado  (1992)  were  among  the  first  to  imegrate  die  power  of  CBT  authoring 
environmoitswithdietedinQlogyof  ITS.  llieir  prototype  system  was  the  harbinger  of  many 
mote  powerful  ambfoadons  of  traditional  CBT  nd  next  generation  ITS  technctiogies.  Most 
reoendy,  DARPA  has  funded  a  unique  consortium  of  Apple  Computer,  textbook  piddishers  arch 
as  Houghton>Mifflin,  and  rrSeaqietts  Beverly  Woolf  and  JdhnAndoson  to  begin  the 
development  of  next-generation  airthcning  tools  for  instruction  and  training. 

Hie  relative  quiescence  of  die  80s  transitioned  into  the  current  state  of  ITS  afiitirs, 
marked  by  a  pmcqition  of  instability  and  oorttroversy. 

1990s:  Great  Delmies 

Ihe  four  hot  ITS  topics  tight  now  may  be  broadly  characterized  as:  (a)  How  much 
learner  coDtrcd  should  be  allowed  in  systems?  (b)  Shoidd  leaineis  imeiBct  with  ITS  individually 
OTOollaborativdy?  (c)  Is  learning  situated,  unique,  and  oogohig,  or  symbotic  uddoes  itftdlow 
an  information-IHOoessing  model?  and  (d)  Does  virtual  reality  (VR)  unkpiely  oontiibute  to 
learning  beyond  CAI,  ITS,  or  even  multi-media?  Tbeie  are,  of  course,  proponents  and  opponents 
to  eadi  of  these  posititms. 

riRyiwft  nf  I  jijiiiifir  Crnitmi  The  debate  over  the  amouDtofleamercontroldutdioold  be 
a  part  of  die  learning  process  has  raged  for  many  years.  On  die  one  band,  some  have  argued  dud 
discovering  information  tm  one’s  own  is  die  best  way  fo  team  (e.g.,  Bruner,  1961).  On  the  other 
hand,  structure  and  direction  have  bear  stressed  as  the  impcntant  fogredienis  in  the  promotirm  of 
smdentteaniing(e.g..  Ausubel,  1963).  The  same  ddMte  has  rqipeaied  in  the  ITS  arena.  Two 
differing  perspectives,  representing  die  ends  of  this  continuum,  have  arisen  in  reqxwse  to  die 
tesue  of  the  optimallTS  learning  envinnment  Oneqiproachistodevelopaconqioterized 
envirmunent  oontainirig  assorted  tools,  and  allow  teamers  fieedom  to  and  leam 
indqmidendy  (e.g.,  (joIUds  &  Brown,  1988;  Sliute,  Glaser,  &  Raghavan,  1989;  White  A 
Horowitz,  1987).  Advocates  of  the  jpposing  position  argue  that  it  is  mote  effective  to  develop 
straightforward  teaming  environments  with  no  digressions  permitted  (e.g.,  Anderson,  Boyle  A 
Reiser,  1985;  Corbett  and  Anderson,  1989;  Sleeman,  Kelly,  Martinak,  Ward,  &  Moore,  1989). 
This  disparity  between  perqiectives  becomes  more  comidicated  because  the  issue  is  not  just 
whidi  is  die  better  learning  environment,  but  which  is  the  b^r  environmem  for  udnm,  a  dassic 
rgidtude-treatment  imeractimi  questirm  (Cnmbach  and  Snow,  1981).  Hiete  are,  undoubtedly, 
temporal  aqiects  to  diis  issue  as  wdL  For  instance,  it  may  be  more  efficient  to  leam  a  new 
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cognitive  skill  initUdly  by  direct  instniction,  then  later,  by  greater  exploration.  In  this  way, 
teamers  can  better  oontiol  dieir  own  learning  process. 

Menin,  Reiser,  Ramey,  and  Ttafton  (1992)  investigaied  how  human  tuton  dealt  with  the 
issue  of  learner  contnd.  They  compared  human-  to  cranputer-tutcning  tedmiques,  and  found  tlutt, 
while  expert  human  tutors  did  sometimes  act  like  modd-traceis,  they  actually  maintained  a 
"ddicate  balance”  betwem  (a)  allowing  stiulaits  freedom  and  control  and  (b)  giving  students 
sufBdentguidanoe.  In  general,  pedagogical  researdi  findings  differ  with  r^ard  to  the  arnoum  of 
teamer  control  to  allow  in  automated  systems  (e.g.,  Fbx,  1991;  Lqrper,  As{»nwall,  Munune,  & 
Qiabay,  1990;  Iterrill,  Reiser,  &  Landes,  1992).  In  addition  to  the  temporal  factor  dted  above, 
this  issue  of  learner  control  is  also  greatly  dqxndoit  on  other  variaUes,  such  as  the  subject 
matter  being  instructed,  die  desired  kmwledge  or  skin  outcome,  incoming  Altitudes,  and  so  on 
(see  Kylkmen  &  Shute,  1989,  fora  comidete  discussimof  diese  imeracting  variables).  That  is, 
if  the  desired  teaming  outcome  is  a  smoothly  executed  skiU,  it  may  be  more  efBdent  to  instruct 
certain  teamirtg  tasks  wid)  direct  instruction  and  {denty  of  practice.  But  if  the  desired  leamitig 
outcmne  is  a  functknal  mental  model  of  relevant  ininciides,  an  exploratory  environment, 
comptete  widi  various  compments  such  as  on-line  drcuits,  ammeters,  and  lesistors,  may  be  tdutt 
is  needed  to  addeve  that  educational  objecdve. 

Most  current  computer-administered  instructkmal  systems  do  not  foster  self-reliance  in 
students,  OT  encourage  diem  to  seek  new  information  (mdieirowa  To  rectify  diis  deficit, 
Batnaid,  Erfcens,  and  Sandberg  (1990)  propound  die  building  of  more  fiexiUe  systems  padraging 
communication  expertise  as  a  sqiarate  component  With  less  teamer  initiative,  it's  mudb  easier 
to  interpret  irqwt,  but  at  vdiat  cost  to  teaming  outcome?  In  Jiqian,  research  is  bdi^conhicted 
along  these  lines.  The  conoqtt  and  devetopmem  of  ITS  is  becoming  merged  with  interactive 
teaming  environments  (ILE)  to  produce  what  is  referred  to  as  a 'hiHuodns  learning  environment" 
(BLE)  (Otsidd,  1993).  Whereas  die  main  strmgdi  of  ITS  is  its  ability  to  derive  a  studem  model 
based  on  die  identification  of  acquired  rules,  its  main  weakness  is  die  inability  to  heb>  learners 
acquire  new  knotdedge  by  themsdves.  In  contrast,  studods  in  an  ILE  can  extract  and 
comprdiend  rates  induced  from  a  complex  domain,  but  die  ILE  cannot  explictdy  idnitify  a 
student's  misconcqitions  or  tutor  diem  in  terms  of  their  comjMehensiontevd.  Thus  the  two  (ITS 
and  EJE )  are  conqriementary  to  me  uiodier,  and  BLE  represents  combining  die  stiengdis  of 
each. 

Anodier  wity  to  increase  learner  control  has  been  suggested  by  Bull,  Pain,  and  Bma 
(1993).  Their  intriguing  alternative  to  traditional  studmit  modeling,  that  of  rqilacing  the  burden 
of  the  ITS,  is  to  produce  accurate  representatims  of  the  teamer's  knowledge  state;  the  learner  is 
onpowered  widi  greater  contnd,  e.g.,  to  construct  and  repair  die  model.  Bull  and  assodaies 
contend  that  their  model  will  result  in  a  more  accurate  representation  of  the  learner's  beliefs,  and 
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tbus  be  mofe  UgUy  regaided  by  the  studott.  IliB  learner  is  expected  to  baiefit  through  die 
leflectkn  necessary  to  aocompUriidiis  modeling  task.  Unfortunately,  no  data  have  yet  been 
ocdlectBd  about  the  eCBcacy  of  Itris  novel  approach. 

"Ooadwd  piactioe  environments'*  (Le..  Sherlock  I  and  II)  lepresott  y^  another  way  to 
provide  contnd  during  kaming  by  combining  apprenticesfaip  training  with  imelligent 
instructkmal  systems  (Liyoie  A  Lesgold,  1992;  Lesgcdd,  Eggan,  Katz,  A  Rao,  1992).  Ihese 
systems  siqiportpealer  learner  initiative  because  the  apprentice  teams  by  doing  (singularly  or 
odlaboiatively);  knoudedge  is  andiored  in  experience;  and  the  coadi  provides  knowledge  within 
an  applicabte  conlexL  imelligent  systems  are  developed  widi  many  of  the  characteristics  of 
human  apprenticedi^  and  performance  can  be  easily  assessed.  Through  replay  and 
comparisons  with  the  expert  performance,  this  approach  also  supports  trainee  analysis  of 
performance. 

Salomon  ( 1993)  supports  the  trend  of  moving  away  from  buildhig  traditional  ITS  and 
towards  the  design  of  systems  as  cognitive  fords.  He  sees  cognitive  fools  manipulated  by 
students  as  instruments  tibat  promote  constructive  thirddng,  transcending  cognitive  limitations, 
and  making  it  possibte  for  students  to  engage  in  cognitive  operations  they  wouldn't  otfawwise 
have  been  cqtebte  of.  Some  ITS  programs  make  most  diagnostic  and  tntmialdecisioos  for  die 
student;  dierefiire  diey  are  not  realty  oogrddve  fords  because,  "diey  are  not  designed  to  iqigtade 
students'  intelligent  engagements."  180).  Abo  in  accordance  widi  die  notion  of  computers  as 
teaming  fords,  teamers  should  have  the  option  to  aher  die  degree  of  contnd  themsdves,  from 
none  (e.g.,  rfidactic  environmerd)  to  maximum  (e.g.,  disoovery  environment),  as  necessary. 

By  ddfUiig  toward  increased  learner  contnd,  are  individuab  ud»  are  not  very  active  or 
exploratory  by  nature  being  penatized  or  handicappetl?  Snte  and  Glaser  (1990)  investigated 
individual  rliflerenoes  in  teaming  from  a  rli80overyenviromient(Smilfatown)  and  ftaroddat 
individuab  who  rtemonstrated  systematic,  exploratory  behaviors  (e.g..  recording  baseline  data, 
limiting  the  mmber  of  changed  variables)  were  significandy  more  successful  in  Smitfatown 
compared  to  those  who  revested  teas  systematic  bdaviors.  Ondiebad8ofdiatfintting,diey 
hypodiesized  in  a  difterent  study  (using  an  etectricity  tutor)  diathigh-eiqdoratory  indivirbuds 
wouki  team  mne  from  an  indnetive  environment  (than  from  a  mote  directed,  applied 
environmem),  and  tess-eaqjkmtory  teaims  would  benefit  from  a  supportive,  ^spited 
environment  (compared  to  an  inductive  one).  A  person's  expiontmy  level  was  quantified  based 
on  certain  indioes  (e.g.,  number  of  tries  and  lengtii  of  time  apeat  dianging  a  lesistiM*  value,  using 
the  on-line  voltmeter  or  ammeter).  Subjects  were  randomly  assigned  to  one  of  two  learning 
environments,  and  the  data  were  analyzed, /mutAoc.  The  hypodiesized  teaming  stylet^  aptitude 
interaction  was  suppmted  by  die  dtea  (Sbuie,  1993-b).  So,  dbeovery  learning  enviimBnems  do 
not  suit  everyone  equally  welL  Ftv  some,  diey  provide  a  teaOy  bad  fit  To  dmeimine  whether 
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diis  Idiid  of  learner  styte  by  treatmem  intenctioo  is  fepUad)ie,  Shale  (19{M)  ocxKiiicied  a 
cw^^mMtniy  test  of  the  Same  AH.  iqxMted  above.  Subjects  were  placed  a  priori  in  one  of  two 
environments  based  on  the  decision  rate  obtained  from  the  previous  study.  And,  in  fKt,  the  ATI 
was  oonflnned. 

hi  conclusion,  a  mid-point  between  too  much  and  too  Uttte  teamer  control  is  piobaUy  the 
best  bet  as  Ctf  as  optimairrs  teaming  environmott  Fuithennore,  this  milestone  should  not  be 
fixed,  but  should  diange  in  response  to  leafners’evtrivhig  needs.  Hnally,  leameis  should  have 
some  inpirt  into  the  design  (rf  the  environment,  m  wen. 

Our  next  ddMle  addresses  the  issue  of  udiedier  teaming  alone  is  better  or  worse  than 
teaming  In  conjimction  with  ottiers  (where  “odieis"  may  mean  other  humans,  or  with  a  oompuim 
acting  as  a '^Dtnei^  in  the  teaming  process).  As  with  eveiyttiing  relating  to  teaming,  there  is 
probably  no  dear  cut  answer  to  this  question;  there  is  no  "ovendt"  superior  way  to  team.  Rather, 
it  is  ahnost  certain  ttiat  interactions  exist,  ariiere  solo  teaming  may  be  siqterior  for  certain  topics 
(e.g.,  rote  measorization  of  mnlti^calion  tdties)  m  for  particular  teamer  ^pes  (e.g.,  hi^y 
motivated  individnals).  Gdlaboralive  teaming  may  be  more  elective  for  other  domains  or 
persons.  White  we  don't  ^tedficany  address  these  interactions  in  the  following  discossions,  they 
should  be  kept  in  mind. 

indiviAMi  vs  nniiihntwh»  Iiafnhw  Traditionally,  ITS  have  been  designed  as  siiigte- 
tearoercmeipriaes.  Bloom  (1984)  and  ottieis  have  presented  compdlingevktaice  tint 
indivktnaH/ed  tutoring  (using  hanan  tutws)  engenders  the  most  effective  and  cfBcient  teaming 
across  an  amy  of  domainB  (see  alK)StBiie& Rician,  1990;  Wodf,  1988).  Ftarthennoie, 
iatieiligenttiilotirig  systems  qpfiamhtftirispriix^jie  of  individualized  instractiotL  fohisoften- 
dted  1984  paper.  Bloom  presented  a  chalfcnge  to  faisiructional  researchers  tiiat  has  been  called 
the 'Two  sigma  probiem."  T1»  goal  te  to  achieve  two  standard  deviation  inqaovcmcanswitti 
tutoring  over  traifitional  instruction  mettiods.  So  te,  this  goal  has  yet  to  be  attained  using 
individoalizedrrs. 

An  altemiuive  approach  to  individoalized  instruction  is  odlabotative  teaming,  ttte  notkm 
that  studenls,  working  togetiier,  can  team  mme  than  by  titemsdves,  eqtedaUy  adien  they  bring 
oomptemmtary,  rather  titan  idenlical,  contribitiions  to  the  joint  enterprise  (Qimmings  &  Sdf, 
1989).  Cktilaboration  is  defined  as  a  process  by  which  "indhrkloals  negotiate  and  share  meanings 
relevaittotheptoblem-s(dvingtaticathand.''(Teastey&Rosdidte,  1993,  p.  229),  and  is 
Satinet  fiom  cooperation  tritich  relates  to  tile  division  of  labmr  required  to  achieve  some  task. 

Two  empirical  questions  relevant  to  this  diapter  indude:  (a)  Ate  two  heads  better  ttum 
one?  and  (b)  Can  intelligeot  computer  systems  support  cdlaborative  teaming  endeavors? 
Recently,  leaeaiditeb^inning  to  tiled  ligltt  on  both  of  tiiese  questions.  For  example,  many 
leaeardtets  have  tiiown  impressive  student  gains  In  knowtet^  and  skin  acquisition  from 


28 


odlaborative  teaming  environmems  (e.g..  Brown  &  Palincsar,  1989;  Lampeit,  1986;  Palincsar  ft 
Brown,  1984;  Scwdmdla.  Beidter.  McLean,  Swaflnw,  ft  Woodniff,  1989;  Sdioenfeld,  1985). 
RntfieraiOTMiie  few  studies  of  tile  efifediveness  01  coUabmative  teaming  in  computer-based 
teamins  enviraomeots  have  dso  been  positive  (e.g.,  Justen,  Waking,  ft  Adams,  1990;  Katz  ft 
Lesgold,  1993;  Papert,  1980). 

Tbeie  are  basically  two  ways  of  imptementii^  coflaborative  teaming  oivironments  using 
oompoten:  (a)  a  small  group  of  teameis  inteiact  with  a  single  intelligent  computer  system,  or  (b) 
the  oompmm’ system  itsdf  serves  as  die  "partner”  in  the  odlabmation.  Ihe  first  wi^CLe.,  a  small 
groiq)  using  one  computer)  iqueaents  an  extension  of  the  research  on  ctdteboradve  teaming  in 
dassrooms.  In  diis  case,  some  of  the  issues  that  need  to  be  addressed  have  been  oufiined  by 
TeasteynidRosclielte(1993).  Ihe  system  must  be  dile  to:  (a)  introduce  and  aooqxkiKndedge 
imo  a  joint  probtem-sdving  spaoe,  (b)  monitw  ongoing  activittes  fw  evidence  of  divergences  in 
meaning,  and  (c)rq)air  divergences  ttiat  impede  die  progress  of  die  odlabmatkn.  Ihe  dUferenoe 
between  dris  list  and  general  modeling  issues  in  ITS  is  that  it  deals  with  a  student  model  that's 
built  upon  a  joint,  rather  dun  singte.  problem  solving  apace.  The  second  way  of  hnptementing 
coilabwatioo(Le.,assigidhg  the  computer  as  the  teamei*s  partner)  represents  an  intriguing  twist 
on  die  notion  of  oolUbontive  teaming*  To  iltnstrate,  Cummings  andSdf(1989)  proposed  a 
coflaborative  intdligent  education  system  (IBS)  that  engages  the  teamer  ins  paitnersfaip.  Here, 
dnooniputerservaasaodUboiaior.iiotasanainfaoiitatianinstnictor.  In  both  cases,  a  student 
model  sdn  most  be  derived,  cidier  dutt  of  an  hKfividwd  or  a  groig). 

Additional  research  and  oontroOed  studies  must  be  conducted  in  Older  to  test  die  rdrttive 
efficacy  of  wdlaborative  versus  intBvklmJized  instruction.  For  a  variety  of  reasons  (e.g.,  greater 
range  of  Aared  knoudedge,  resooroe  limitations,  etc.),  the  notion  of  conaboiative  teaming 
environments  is  appealing.  Ihere  are  a  lot  of  unanswered  research  questions  that  need  to  be 
addressed,  however.  Some  (rfthese  (listed  in  Katz  ft  Lesgold,  1993)  indude:  What  parts  of  die 
cunlcolnm  should  be  teamed  odteboradvdy,  rod  uhat  parts  teamed  individoally?  What 
teaching  mediods  Aould  be  used  to  achieve  die  imtinctional  goals,  and  how  should  they  be 
sequenced  to  optimize  teaming?  What  should  die  computer  tutcMr  do  white  students  work  on 
probtems?  What  additional  rotes  could  the  oomputo*  coach  perform?  lUs  area  of  research  is 
also  fikely  to  shed  tight  on  die  interactions  mentioned  earlier.  Wenowpiesemthediirdhot 
topic,  namdy,  die  ntenie  of  teaming  and  its  impact  on  ITS  design. 

■UHiMteH  1  ^fiiiny  rmnipvMsy.  To  suppoTtets,  tiUs  is  not  just  a  trend,  but  a  radically 
new  perspective  (or  pidtoaopt^)  tint  siqipoits  the  int^ration  of  "...psychological  theories  of 
physical  and  cognitive  dcills,  uniting  emotions,  reasoning,  and  development,  in  a 
neurobMogically  grounded  way."  (dancey,  1993,  p.  98).  It  has  also  been  referred  to  in  the 
ttteratuie  as  "situated  action"  and  "sitoated  cognition."  Recendy,  several  prominent  journals  have 
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devoted  entire  issues  to  die  detrete  ooncenidig  the  value  of  situated  leatning  compared  to  die 
more  standaid  paradigms  (e.g.,  ACT*.  SOAR):  1993  Chynitive  Science.  17(1).  and  1993  Jmimal 
of  Artficiri  liUriliBwirf.  and  PAimrinn  A/U 

Obviously,  one's  belief  in  eidier  situated  cognition  or  the  traditional  infonnatkm- 
piocessiiig  modd  has  imfriicatioiis  for  the  design  of  ITS.  To  illustrate  this  distinctitm,  first 
consider  Oreeno's  summary  of  situated  cognititm's  petspective  on  where  knowledge  resides: 
Hather  than  thinkii^  diat  knowledge  is  in  die  minds  of  individuals,  we  could  alternatively  tfiink 
of  knoudedge  as  the  poiratiat  for  situaied  activity.  On  diis  view,  knowlet^  would  be 
understood  as  a  relatkm  between  an  individual  and  a  social  or  physical  situation,  rather  than  as  a 
piopeityofaniiidividnaL''(Qieeno,  1989,  p.  286).  Next,  consider  the  naiuie  of  knowlet^  from 
die  infofmMion>piooeasi08  perqiective.  Anderson's  (1983)  ACT*  dieory  proposed  two 
fondamental  fonns  of  knov^et^:  procedural,  represented  in  the  fonn  of  a  producticm  system, 
■id  decfonitfve.  represented  in  the  fimn  of  a  node-litdcnetwmk  of  propositions.  Bodi 
representations  are  believed  to  operate  widdn  Ung-tenn  and  shoit-tmm  memory  structures. 

Hieae  two  positions  present  quite  diCEerent  views  on  how  learning,  or  knowledge 
aoquteition.  oocufs.  bi  the  first  case  (situated  cognition),  teaming  is  a  process  of  creating 
repreacntadons.  invenfing  languages,  and  formulating  modds^r  the  first  dme.  Leamhigis 
ongoing,  occuning  widi  every  dmught.  percqition,  and  action,  and  is  situated  in  each  unique 
circumstance.  Situaied  cognition  argues  for  n  instructional  sy^em  rich  with  explicit  tools  and 
varied  exempims  that  can  supfiort  and  extend  levnets' discovery  processes.  Insight  is  more 
likely  udien  die  problematic  sttuation  is  so  arranged  that  all  necessary  aqiects  are  open  to 
observation."  (Bower  dtHilgaid,  1981,  p.  319). 

Hie  second  position  (information  processing)  sees  teaming  as  progressing  from 
declarative  knoudedge,  to  procedural  skills,  to  antomatic  skills,  dqiendeitt  vpoa:  enddeis  (i.e., 
what  one  tdrearty  knows  and  can  transfer  to  new  tituations)  and  mediators  (i.e.,  cognitive 
processes  determining  what  one  can  acquire,  such  as  woriting-memory  ogncity  and  informatitm 
processing  qieetfi  (e.g.,  Anderson,  1983, 1987;  Kyllonen  ft  Christai,  1990).  Ihus,  teaming 
refers  to  the  addition  and  restructuring  of  information  to  a  database,  in  accordance  widi  specific 
learning  medusiiams  (e.g.,  knowledge  compilation,  transfer).  To  facilitate  learning,  tme  must 
build  a  system  that  cm  (a)  Analyze  die  initial  state  of  knowledge  and  skill;  (b)  Desoibe  the 
desired  OT  end  state  of  knowledge  and  ddll  (teaming  outcome);  and  (c)  Present  materitd  and 
probiemsdiat  will  tmsHion  a  teamer  from  initial  to  desired  state.  Ihis  kind  of  tutoring  sy^em 
is  based  on  a  well-defined  curriciitum  that's  been  so  arranged  to  promote  knowtedge/kkill 
aoqitisition  (OT  fiteilitate  trmsition  fiom  curiem  fo  goal  stale). 

R  may  be  diatdiese  two  positions  are  mutually  exdusive.  That  is,  knowle^  eidier 
resides  imemally  in  one's  head,  or  externally,  in  die  environmenL  Altemativdy.itmaybethat 
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there  is  some  oveflq>,  wherein  some  fonns  of  knowledge  are  stored,  anu  some  d«1  vable  fnnn 
die  comat  situation,  hi  a  preUminary  attempt  to  bridge  the  gap  between  siuutted*  and 
traditioiud-leaiiiing  modds,  Suite,  Oswlick-Orendell,  and  Yotuig  (1993)  have  recently  developed 
a  aeries  of  statistics  modules,  StatLady.  Learning  is  situated  within  various  gaming 
envirotments  (e.g.,  ”Stat  Cnps”);  the  theoretical  postulates  are  that  learning  is  a  constructive 
process,  eidianced  by  experiential  invrdvement  with  the  subjea  matter,  that  is  situated  in  real- 
world  examples  and  probtans.  Furthermore,  the  system  has  a  well-defined  curriculum  in 
accordance  with  popular  learning  dieory. 

Aoomding  to  constructivism,  fearners  actively  construct  new  knowledge  and  skills,  either 
fiom  whatdiey  already  know  (infoimatioii-processing  premise)  or  from  what  resides  in  the 
enviromnent  (situated  oration  stance).  Bothpositions  would  probably  agree  that  learners  do 
not  come  to  a  teaming  dtuatkn  widi  a  tabula  rasa,  but  rather,  as  active-pursuers  (not  passive- 
redplents)  of  new  knowledge  (e.g.,  Bartlett,  1932;  Collins,  Brown,  &  Newman,  1989;  E)resdier, 
1S191;  Edetanan,  1987;  Piaget,  1954).  Bodi  positions  also  siqjpott  die  positkm  that  die 
construction  process  can  be  enhanced  environmems  suppmting  eiqieiiendal  teaming. 

Researdi  in  this  area  has  shown  diat  knoudedge  derived  experientially  tends  to  be  more 
memorable  dam  passively-ieoeived  knowledge  because  the  experience  C'doing"  rather  dian 
"receiving")  provides  cognitive  structure,  and  is  intrinsically  motivating  and  involving  (e.g., 
Rtedmn  &  Yarbroi^  1985;  Harel,  1991;  Harel  &  Pqmt,  1991;  Shute  &  (Baser,  1991; 
^pencerA  Van  Eynde,  1986).  Rrially.udien  instruction  is  situated  (mrandimed)  in  iriteresting 
and  real-worid  probtem-solving  scemuios,  that  also  is  believed  to  enhance  teaming  (BrorAs, 

1991;  Brown,  C^tdlins,  &  Duguid,  1S189;  dancey,  1992;  Ccdlins,  Brown,  &  Newman,  1989;  Lave 
&  Wenger,  1991;  Sudunan,  1987;  Ihe  Cognition  A  Technology  OroiB)  at  Vanderbilt,  1992). 

Ihe  Cognition  and  Technology  Group  at  Vanderbilt  (1992)  has  also  been  working  on 
developing  a  pedagogical  qiproacfa  to  situated  cognitkm.  They  define  "andwred  instruction"  as 
an  attenqjt  to  actively  engsge  teanrers  in  die  teaming  process  by  situating  instructxm  in 
interesting  and  real-wmldpioblem-solvir^enviroimaits.  Ridier  than  teaddrig  students  how  to 
solve  particular  proMems,  diese  systems  teadigmeralizaMe  dtiUs.  hebiful  across  a  variety  of 
pioUem-srdving  situations.  Ihemajorgoalofdiistypeofinstructionistocieateaudientic- 
tedirig  environments  in  which  one  can  ex|dore  and  undnstand  problems  and  opportunities 
mqKrienced  by  erqierts  in  a  domain,  and  team  dxHit  the  tools  these  eiqierts  use.  Thisgrouphas 
developed  a  series  of  advomues  for  middle-school  studems  focusing  on  math  proUem 
formulation  md  pmUem  solving.  These  are  the  "Adventures  of  Ja^r  Woodbury"  series.  The 
g  of  the  project  is  to  facilitate  broad  tiaiKfer  to  odier  domains,  embodying  several  design 

prindides:  (1)  video-based  presoUatimi,  (2)  narrttive  fcmnat,  (3)  generative  learning,  (4) 
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onbedded  data  design.  (3)  (noUan  oomidexity.  (6)  pairs  of  related  advemures,  and  (7)  links 
across  die  curriculuni. 

One  of  die  majcn’  proMems  with  this  whole  debtte  over  situated  cognition  versus 
tiaditkxud  infoimadon  processiiig  models  is  that  the  fbnner  position  sunfdy  has  not  tested  its 
undeilying  hypotheses  at  this  dme,  while  the  latter  has  enjoyed  decades  of  solid  research.  Vera 
and  Simon  (1993),  rebutting  Clmcey's  support  pqKifs)  for  sinutted  learning,  stated,  "Clancey 
leaves  os  with  philosophy  (vtiiedier  correct  or  ncH  is  another  matter),  but  widi  precious  little 
science."  (p.  118).  And  that  appears  to  be  true.  Because  cognitive  psychology  is  an  eminrical 
science,  studies  need  to  be  conducted  that  examine  claims  made  by  any  new  positiott  For 
instance,  supporters  of  our  final  "hot  topic"  of  the  90's  (Virtual  Reality,  or  VR)  claim  that  this 
new  techiKdogy  can  improve  learning  by  virtue  of  fully  immersing  die  learner  in  the  learning 
process  (teaming  by  saturation).  Butisthereany  veracity  to  this  claim?  It  is  certainly  testaUe. 
Hie  relationship  between  expertenoe,  teaming,  and  pedagogy  is  a  Mar  patch  of  thorny  tprestkxK. 
Recent  dieoretical  harartgues  on  die  rutture  of  situated  learning  have  laid  a  kind  of  groundwmk 
for  VR  by  arguing  for  an  qjisiemtdogy  of  teaming  based  on  operience. 

VirtiMl  Rftrihy  and  I  jamhiy.  A  collection  of  techncdogies,  known  as  Virtual  Reality 
(VR).  has  reoendy  been  exciting  the  instructional  technology  community.  This  new  technology 
refers  odtectively  to  die  hardware,  software,  md  interfece  tectuKdogies  available  to  die  user 
interested  in  erqieriencing  certain  aspects  of  a  simulated  3-dimensional  environment  Hie 
simutated  aspects  of  the  envirorunent  ("worfcT)  conendy  inciude  a  stereoscopic,  low-to^edium 
fiddity  visual  rqxesentationdiqdaiyed  on  a  head*mouiileddiqday  system.  Using  head-tracking 
technologies,  one  can  update  die  diqday  in  accordance  with  head  and  body  motimis.  Hiis 
feature,  along  widi  the  stereo  disparity  of  the  hnages  on  the  two  screens  (one  for  each  eye), 
sipptnt  the  illusion  of  moving  around  in  3-dimensiand  qMce. 

Unqpiestionably,  VR  chnges  die  relationdiips  between  learning  and  eiqietience, 
highligfating  die  rote  of  perception  (particularly  visual),  in  leaning.  Experience  is  bodi  social 
and  perceptual,  and  VR  epitomizes  the  notkm  of  experiential  teaming.  Many  systems  are  now 
being  developed  that  have  demonstrated  the  success  of  the  experieittial  approach.  Thecurrent 
questionis:  Does  VR  represent  the  next  logical,  developmemal  step  in  the  design  of 
instructional  systems?  In  odier  words,  does  die  immersion  experience  (i.e.,  extra  fidelity  and 
related  cost)  significaittly  improve  learning  and  pofmmance  beyond  the  more  traditional 
pedagogical  approadies? 

Reoendy,  there  have  been  some  empirical  data  collected  on  the  relative  success  of  VR  in 
terms  of  instractional  effectiveness,  as  wen  as  skill  transfer  to  the  real  world.  Forinstance, 
Regian,  Shdtildce.  and  Mtmk  (1992)  showed  tint  peofde  can,  indeed,  leam  to  perform  certain 
raskK  from  virtual  enviraiments  (e.g.,  omsote  operati(»K  and  large-scale  spatial  luivigation). 
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Next,  kiiowle<lge  and  akill  acquired  in  a  VR  have  been  shown  to  transfer  to  peifonnanoe  in  the 
real  wmld.  RegUai,  ShdMlske,  and  Monk  (1993)  found  tiuu:  (a)  VR  console  operadtxis  training 
can  transfer/fiacilitate  real  world  console  operadons  perftmnance,  and  (b)  VR  spatial  navigatitm 
training  successfully  transfers  to  real-world  qntial  navigatiort  In  coturast  to  the  Regian,  et  al. 
(1993)  findir^,  however,  those  r^rted  by  Kozak,  Hancock,  Arthur  and  Chrysler  (1993) 
showed  no  evidence  for  transfer  of  a  "pick  and  [dace"  task  from  VR  to  the  real  world.  However, 
die  criterion  task  used  in  that  study  was  cpiite  easy;  thus,  die  conclusions  may  actually  be 
inoonclusive.  So,  even  with  die  reladvely  poor  fiddity  and  interface  currendy  availabie  in  VR 
tedmcdogy,  there  is  some  evidence  for  its  efficacy  and  potendal  as  a  serious  leaming/trainirig 
enviromnent 

Another  positive  example  of  VR's  potradal  for  training  was  presented  by  Psotka  ( 1993) 
idio  argued  diM  VR  creates  one  uniform  point  of  view  on  any  representation  that  overcmnes  die 
conflicts  and  cognitive  load  of  maintaining  two  disparate  points  of  view  (Swdler,  1988).  The 
reduced  cognitive  overhead  resulting  from  the  single  "egoccnter"  in  a  VR  should  expedite 
infiMmation  access  and  learning.  Central  to  this  penxptualeiqKrience  of  VR  is  the  pooriy 
understood  phetKmenon  of  immersion  or  presence.  Preliminary  insight  based  on  the  SIMNET 
erqieiience  (Psotka,  1993)  provides  not  only  persorud  testimonials  to  die  motivating  and 
stimulating  effects  of  the  social  and  vducle-based  Inunersion  of  synthetic  environments,  but  also 
preliminary  efifectiveness  data  on  its  potency  for  learning  and  trainmg.  That  is,  even  though 
SIMNET  provides  n  inqmveridied  perceptual  simolation  of  a  tank  in  action,  the  cues  fmn 
interactive  oommunicadons  among  crew  members,  as  well  as  the  auditory  and  visual  cues  of  the 
simulated  sights,  provide  gut-wrenching  and  sweaty  believability.  What's  more,  the  evidence 
dearly  shows  alevd  of  training  efifectiveness  (even  widiout  acutiicuhim)  dua  is  siqierior  to 
many  odierdasstoom  and  simolation-based  efforts  (Bessoner,  1^1).  Research  is  continuing  on 
how  to  mdce  dus  trainir^  more  effective  by  including  surrogate  crew  members  and  intelligent 
semiautomated  forces  in  the  environmertis.  The  need  to  involve  dismounted  infantry,  not  just 
tardcs  and  vdiides,  is  cteatir^  a  research  base  for  better  computatimud  models  of  agoits  and 
coaches  (Badler.  Phillips,  and  Webber.  1992). 

Virtual  reality  diows  promise  in  die  construction  of  microworlds  for  ph^ics  and  odier 
science  instruction.  Fbr  instance,  Loftin  and  Dede  (1993)  are  creating  a  Virtual  Physics 
Laboratory  fitom  die  base  facilities  of  a  VR  world  created  for  NASA  astronaut  training.  Indieir 
virtual  laboratory,  students  can  ccmduct  experiments  in  a  virtual  world  where  everyday  acddents, 
structural  imperfections,  and  extrinsic  forces,  such  as  hiction,  can  be  completely  controlled  or 
riimiiMitgri-  Balls  that  bounce  widi  com[d^?  determirusm  can  be  measured  accurately  at  all  times 
and  [daces,  and  can  even  leave  visible  trails  of  dieir  paths.  The  effects  of  gravity  can  be 
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contndled,  and  variatkMis  of  gravity  can  be  expertenced  visually,  and  peitiaps  even 
Idnestheticany. 

Ahtwugh  the  perceptual  aspects  of  experience  are  cleariy  important,  it  is  easy  to  assume 
ttiat  diere  are  no  difficulties  to  learning  from  existing  visual  representations  and  simulations,  like 
phraographs,  graphs,  snd  static  drawings.  It  is  easy  to  downfday  and  overtook  difficulties  in 
modem  learning  envircmments.  Most  of  us  are  experts  at  inteipreting  visual  reinesentatimis  on 
{Minted  pages  (figures,  gr^)hs,  photogr^ihs,  ioms,  drawings,  and  prints),  but  it's  easy  to  forget 
the  difficulty  we  once  experienced  as  we  tried  to  interpret  scatter  plots  and  line  grrq^.  We  know 
from  many  studies  that  those  difficulties  never  oMnpletely  go  away.  For  younger  learners,  they 
may  be  evoi  more  {nmiounced.  VR  can  remove  these  difficulties  to  a  degree  and  make 
information  more  accessiUe  duou^  the  evolutkMiaiily-iMepared  channels  of  visual  and 
perceptual  mqwrienoe.  As  to  the  question  of  tiriiether  the  delivered  "bang"  is  worth  the  bucks,  the 
jury  is  still  out 

We  now  turn  mr  atterttkMi  away  from  diese  cmitroversies,  and  toward  the  analysis  of  a 
collecrimi  of  ITS  ttiat  have  been  systematically  evaluated  and  reported  in  the  literature.  The 
purpose  of  this  section  is  to  provide  a  flavor  for  evaluations  that  have  been  conducted,  rather  than 
to  review  an  possible  evaluattmis. 
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ITS  EVALUATIONS 

BtdUUng  a  tutor  and  not  evaluating  it  is  like  building  a  boat  and  not  taking  it  in  the 
water.  We  find  the  evaluation  as  exciting  as  the  process  of  developing  the  ITS.  Often, 
the  results  are  surprising,  and  sometimes  they  are  humbling.  With  careful  experbnental 
design,  they  will  always  be  informative.  (Shute  &  Regian,  1993,  p.  268). 

Which  systems  instnict  effectively?  What  makes  them  effective?  One  might  think  that 
incieasiiig  the  perstmalizatimi  of  instniction  (e.g..  model-tracing)  would  enhance  learning 
efficiency,  and  in  the  inocess,  improve  both  the  rate  and  quality  of  knowledge  and  skill 
acquisition.  But  results  cited  in  the  literature  cm  learning,  in  rdation  to  increased  cmnputer 
adqttivity,  are  equivocal.  In  some  cases,  researchers  have  rqwited  no  advantage  of  error 
remediation  in  relatimi  to  learning  outcome  (e.g.,  Bundersmi  &  Olsen,  1983:  Sleeman,  Kelly, 
Martinak,  Ward  &  Moore,  1989).  In  others,  some  advantage  has  been  reported  for  more 
personalized  remediation  (e.g.,  Andersrm,  Gmrad  &  Corbett,  1989;  Shute,  1993-a;  Swan,  1983). 

If,  however,  more  researchers  conducted  controlled  ITS  evaluatitms,  this  issue  would  be 
easier  to  resolve.  But  in  ackfititm  to  die  availalnlity  of  relatively  few  repotted  evaluations  of 
ITS,  there  has  been  little  agreement  upon  a  standard  approach  for  designirig  and  assessing  these 
systems.  Results  from  six  ITS  evaluations  will  now  be  ixesenled. 

SixITSEvahnititins 

A  few  nam|des  of  systematic,  omtrolled  evaluations  of  ITS  reported  in  die  literature 
indude:  die  USP  tutor  (e.g.,  Anderson,  Fatidl,  &  Sauers,  1984)  instructing  LISP  programming 
skills;  Smiditown  (Shute  &  Olaser,  1990, 1991),  a  discovery  wodd  that  teaches  sdendfic  inquiry 
skills  in  the  context  of  microeconomics;  Sherlock  (Nichols,  Pdtomy,  Jmies,  Gott,  &  Alley,  in 
prepaiadmi;  Lesgold,  Lajoie,  Bunzo  &  Eggan,  1992),  a  tutor  for  avicmics  troublediooting;  Bridge 
(Btmar,  Cunningham,  Beatty,  &  Weil,  1988;  Shute,  1991)  teadiing  Pascal  programming  skills; 
Stat  Lady  instructing  stadsdcal  procedures  (Shute  &  OawUck-Grendell,  1993),  and  die  Geometry 
tutor  (Anderstm,  Boyle  &  Yost,  1985),  providing  an  envinmment  in  which  students  can  |uove 
gemnetry  dieorans.  Results  frcun  these  evaluations  dww  that  these  tutors  do  accelerate  learning 
widi,  at  the  very  least,  no  degradatirm  in  outcmne  performance  compared  to  iqrpropriate  cmitrol 
groiqis. 

The  I  J.SP  tutor.  Anderstm  and  his  colleagues  at  Camegie-Mellon  University  (^iderson, 
Farrell,  &.  Sauers,  1984)  developed  a  LISP  tutor  vdiich  provides  students  with  a  series  of  USP 
pmgmmining  exerciscs  and  tutorial  assistance  as  needed  during  the  soluticm  process.  Inone 
evaluation  stu^,  Anderstm,  Boyle,  and  Reiser  (1985)  repotted  data  from  three  groups  of 
mdijects:  faumaiHutored,  cmnputer-tutored  (LISP  tutor),  and  traditional  instruction  (subjects 
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solving  i»oUems  on  their  own).  The  time  to  anniAete  identical  exercises  weie:  1 1.4, 15.0,  and 
26  J  hours,  respectively.  Riithennoie,  all  groups  performed  equally  well  on  the  outcome  tests  of 
LISP  knowledge.  A  secmd  evaluation  study  (Anderson,  Boyle  &  Reiser,  1985)  compared  two 
groups  of  subjects:  Students  using  foe  LISP  tutor  and  students  completing  the  exercises  on  their 
owa  Both  received  the  same  lectures  and  reading  materials.  Findings  showed  foitt  it  took  foe 
gio<q>  in  the  traditioiial  instructicm  ccmdition  30%  longer  to  finish  the  exercises  than  the 
computer-tutored  group.  Moreover,  foe  computer-tutored  group  scored  43%  higher  on  foe  final 
exam  than  the  crmtrol  group.  So,  in  two  difierent  studies,  compared  to  tradifirmal  instmctirm,  the 
LISP  tutor  was  apparetttly  successful  in  promoting  faster  leamit^  with  no  degradation  in 
outcome  performance. 

In  a  third  study  using  the  LISP  tutor  to  investigate  individual  differences  in  learning, 
Anderson  (1990)  found  that  rtfoen  prior,  related  experience  was  held  constant,  two  "meta-factors" 
emerged.  These  two  meta-factors,  or  basic  learning  aUlities,  included  an  acquisition  factor  and  a 
retention  factor.  Not  only  did  these  two  factors  exi^ain  variance  underlying  tutor  performance, 
foey  also  significantly  predicted  performance  on  a  prqjer-and-pencil  midterm  and  final 
examination. 

A  fourth  study  with  foe  LISP  tutor  concerns  foe  usefulness  of  productions  for  analyzing 
learning.  In  arulyzing  student  performance  <m  foe  first  six  ptoUems  in  diapterforee  of  the  LISP 
tutor,  Anderson  (1993,  p.  32)  discovered  uneven,  unsystematic  trends  in  teaming.  CXie  proUem 
was  reladvdy  easy  and  the  next  might  be  relatively  more  difficult  However,  by  decomposing 
foe  {xoUems  into  their  constituent  production  rules,  Anderson  was  aUe  to  ctmvert  foe  duos  of 
foese  results  into  very  systematic  program  solutitm  learning  curves,  for  both  time  and  accuracy. 
He  atulyzed  performance  on  individual  production  rules  across  problems.  Because  producticms 
were  reused,  and  ofoers  newly  introduced  in  eadi  problem,  be  could  plot  perfonnance  in  terms  of 
foe  rrumba*  of  opportunities  eadi  {uoducdmi  rule  had  for  contributing  to  an  addititmal  unit  of 
USP  code.  This  simidifying  transformation  demonstrates  that  knowledge  is  acquired  in  terms  of 
production  rules,  and  that  if  we  are  to  understand  how  learning  cognitive  skills  is  to  be  explained, 
our  atulysis  of  the  task  and  data  ought  to  be  (xmducted  in  terms  of  productitm  rules. 

.Smithtown  Shute  and  Olaser  (1991)  develt^red  an  ITS  designed  to  iminove  an 
infovidual's  scientific  inquiry  skills  within  microworld  oivirmiment  for  teaming  princiides  of 
basic  tttiaoecmKMnics.  In  one  study  (l%ute,  Glaser  &  Raghavan,  1989),  three  groups  of  subjects 
were  compared:  a  group  interacting  with  Smithtown,  an  introductory  economics  dassrornn,  and  a 
ccmtrol  groiq).  The  curriculum  was  identical  in  both  treatment  groups  (i.e.,  laws  of  supfdy  and 
demand).  Results  showed  that  while  all  three  groups  performed  equivalently  on  the  pretest 
battery  (around  50%  cotiea),  foe  dassromn  and  foe  Smithtown  groups  showed  the  same  gains 
from  pretest  to  posttest  (26.4%  and  25.2%,  respectively);  they  significantly  outperformed  the 
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cxMttnd  groiq>.  Althoui^  the  classroom  gmq)  received  more  than  twice  as  much  exposure  to  the 
subject  mttier  as  did  tte  Smithtown  group  (1 1  vs.  S  houfs,  respectively),  ttre  groups  did  not 
differ  on  their  posttest  scores.  These  findings  are  particularly  interesting  because  the 
instructional  focus  of  Smiditown  was  not  (m  economic  knowledge,  per  se.  but  rather  cm  gennal 
scientific  inquiry  skills,  such  as  hypothesis  testing. 

Sh^rinrk  "Shetfock"  is  the  name  given  to  a  tutor  which  provides  a  coached  practice 
enviraunent  for  an  electronics  troubleshooting  task  (Lesgold,  Lajoie,  Bunzo,  and  Eggan,  1990). 
Tbe  tutor  teadies  trouUeshooting  procedures  for  proMons  associated  with  an  F-IS  manual 
avitmics  test  statkm.  Tbe  curriculum  consists  of  34  trouUeshootit^  scenarios  with  associated 
hints.  Astudy  was  OMiducted  evaluating  Sherlock's  effectiveness  using  32  trainees  horn  two 
sqnrate  Air  Force  bases  (Nidxris,  Pokomy,  Jones,  Gott,  &  AUey,  in  preparation).  Pre>  and  post¬ 
tutor  assessmertt  used  verbal  troutdeshooting  techniques  as  well  as  a  p^rer-and-pencilte^  Two 
groups  of  subjects  per  Air  Fbtce  base  were  tested:  (1)  subjects  receiving  20  hours  of  instruction 
on  Sherlock,  and  (2)  a  control  group  receiving  tm-the-job  training  over  the  same  period  of  time. 
Statistical  analyses  indicated  that  there  were  no  differmices  between  the  treatmem  and  tiie  contrtd 
gtoiq»  on  tile  pretest  (means  s  56.9  and  53.4,  reflectively).  However,  on  the  verbal  posttest  as 
well  as  the  pfier-and-pencil  test,  tiie  treatment  group  (mean  s  79.0)  performed  significantly 
better  tiian  tiie  cmtrol  groiqi  (mean  s  S8.9)  and  equivalent  to  experienced  tedmidans  with 
several  years  of  on-the-job  eiqietience  (mean  =  8Z2).  Tbe  average  gain  score  for  the  grmqi  using 
Sietlock  was  equivdent  to  almost  four  years  of  experience. 

its  ("Bridge").  An  intelligent  programming  tutor  was  developed  to  assist  novice 
programmers  in  their  designing,  testing,  and  imidementing  Pascal  code  (Btmar,  camnin^iam. 
Beatty,  &  Weil,  1988).  Tbe  goal  of  this  tutor  is  to  promote  conceptualization  of  programming 
crastructs  or  "{dans"  using  intermediate  solutims.  A  study  was  conducted  with  260  subjects 
tdw  spent  iqi  to  30  hours  learning  from  the  PascallTS  (see  Snite,  1991).  Learning  effidency 
rates  were  estimated  from  the  time  it  to(dt  subjects  to  complete  the  curriculum.  Tbismeasure 
involved  both  speed  and  accuracy  since  subjects  could  not  proceed  to  a  subsequent  protdem  until 
tiiey  were  comidetdy  successful  in  tiie  current  one.  To  estimate  learning  outcome  (i.e.,  the 
breadtii  and  dqitii  of  knowledge  and  skills  acquired),  three  criterion  posttests  were  administered 
measuring  retention,  fifdicatioii,  and  generalization  of  programming  skills. 

Tbe  Pascal  curricolum  embodied  by  the  tutor  was  equivalent  to  about  1/2  semester  of 
introductory  Pascal  That  is,  the  curriculum  equaled  about  7  weeks  or  21  hours  of  instructitm 
fiiiMt  Adding  two  hours  per  week  for  computer  laboratory  time  (conservative  estimate),  the  total 
time  fient  learning  a  half-semester  of  Pascal  tiie  traditioiud  way  would  be  at  least  35  hours.  In 
the  study  discussed  above,  subjects  comfrieted  the  tutor  in  considerably  less  time  (i.e.,  mean  =  12 
hours,  SD  s  5  hours,  normal  distribution).  So,  on  average,  it  would  take  about  three  times  as 
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long  to  team  the  same  Pascal  matwiai  in  a  traditicmal  classroom  and  laboratory  envinxmient  as 
with  diis  tutor  (i.e.,  33  vs.  12  hours). 

While  dl  subjects  finished  the  Pascal  ITS  curriculum  in  less  time  compared  to  time 
needed  to  comptete  the  cuiricuhim  undn*  traditional  instmcdmud  mediods,  there  were  huge 
differences  in  learning  rates  Sound  at  the  end  of  the  tutor.  Fbr  these  subjects  (having  no  prior 
Pascal  erqwrienoe),  the  maximum  md  minimum  completion  times  were  29.2  and  2.8  hoots,  a 
range  of  more  flun  10:1.  In  addition,  sriiile  all  260  subjects  successfully  solved  the  various 
programming  (uotalems  in  the  tutor's  curriculum,  ttieir  learning  outcome  scores  reflected  differing 
d^rees  of  achievement  Hie  mean  of  the  three  olteritm  scores  was  SS.8%  (SD  -  19,  normal 
distribution).  Hm  range  from  the  highest  to  the  lowest  score,  96.7%  to  17.3%,  r^resented  large 
between-^ubject  variation  at  die  cmcluskm  of  die  tutor.  To  accomit  for  these  individual 
differences  in  outcome  performance,  Shute  (1991)  fimnd  that  a  measure  of  worldrtg  memory 
crqiacity,  specific  problem  solving  abilities  (i.e..  [UoUem  identification  and  sequendng  of 
dements)  and  some  teaming  style  measures  (i.e..  asking  for  hints  and  rotming  programs) 
acoourded  for  68%  of  die  outcome  variance. 

StetJjdg.  Two  studies  have  been  conchicted  to  date  with  Star  Lady.  One  study  (Shute, 
Oawiick-Qienddl,  ft  Young,  1993)  tested  the  efficacy  of  teaming  PRC»AmnYfnmi  Star 
in  rdatkm  to  a  traditional  Lecture  and  a  no-treatment  CJcmtrol  group.  Results  showed  that  bodi 
treatment  groiq»  teamed  significandy  more  duui  the  oxitrd  group,  yet  tibere  was  no  difference 
between  die  two  treatment  groiqis  in  terms  of  pretest  to  posttest  improvemerns  after  three  hours 
ofinstrucdoiL  Hk  results  woe  viewed  as  very  encouraging  because,  not  only  was  die  tecture  a 
more  fiuniliar  teaming  environment  fiv  these  sidijects,  but  die  professor  administering  the 
Lecture  had  more  dian  20  years  experience  teaching  this  subject  matter  addle  this  was  Stat  Lady’s 
first  teaching  assignmott.  When  test  items  were  sqMtated  into  dedarative  and  procedural 
categories,  they  found  diat:  (a)  students  using  Stat  Lady  acquired  significantly  mme  dedarative 
knowledge  dian  the  odier  groiqis,  but  (b)  when  procedural  skUl  actpiisition  was  assessed,  the 
Lecture  groiqi  prevailed.  Fmally,  a  significant  qititude-treatment  interaction  was  obtained  where 
high-qititude  subjects  teamed  significandy  more  frian  Stat  Lady  than  from  the  Lecture 
envimiment,  but  for  low-aptitude  subjects,  there  was  no  difference  in  teaming  outcome  by 
conditioiL  Togedier,  these  results  suggest  dud  a  teadier-computercomldnatimi  maximizes 
learning. 

The  second  study  (Shute  ft  Gawlick-Orendell,  1994)  compared  learning  from  Stat  Lady 
vs.  teandng  fiom  apaper-and-pencil  Woikbodt  version  of  the  identical  curriculum,  and 
addressed  die  questitm:  ”What  does  the  cranputer  contribute  to  learning?"  Rndings  showed  diat 
Stat  Lady  teamers  performed  at  least  as  well  (and  in  smne  cases,  much  better)  rni  the  ouUxxne 
tests  compared  to  die  Workbotdt  group,  again  despite  the  {xesence  of  factors  strongly  favoring 
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die  traditional  condition.  Spedficidly,  they  found  that  (a)  Stat  Lady  was  deady  die  superior 
envinxment  for  hi^qidtude  subjects,  (b)  Stat  Lady  subjects  acquired  significandy  more 
declaittive  knowledge  dian  die  Workbook  subjects,  uid  (c)  regardless  of  i^tude,  the  majority  of 
leamas  found  the  Stat  Lady  conditkm  to  be  significandy  more  enjoyaUe  and  helpful  dian  the 
Workbook  conditkm. 

Andermn's  Geometry  •nnor.  The  gemnetry  mtor  (Anderson,  Boyle  &  Yost,  1985) 
provides  SI  oivironmeiit  for  students  to  prove  geometry  theorems.  The  system  mrmittHS  student 
performance  and  jumps  in  as  sorm  as  amistake  is  made.  The  skill  this  system  imparts  is  how  to 
prove  geometry  theorems  ths  smneone  else  has  {xovided.  Schofield  and  Evans-Rhodes  (1989) 
cmdocted  a  large'Scale  evaluation  of  the  tutor  in  place  within  an  urban  high  school.  Six 
geometry  classes  were  instructed  by  the  tutor  (in  conjuncdon  widi  trained  teadiers),  and  diree 
control  geometry  classes  taught  geometry  in  the  tradidmialmatmer.  The  researchers  closely 
observed  die  classes  using  die  geometry  mtor  and  traditional  instructiim  for  more  than  100  hours. 
One  of  the  really  nice  «id  intriguing  results  of  Schofidd  and  Evans-Rhodes  (1989)  evaluation  of 
dds  tutor  was  die  counter-intuitive  reversal  of  its  effects.  Aldiough  the  geometry  tutor  was 
designed  to  individualize  instructirm,  one  of  its  pragmatic  and  unintended  side  effects  was  to 
encourage  students  to  diare  their  eiqietiences  and  coopetativdy  solve  problems.  Sincedieir 
eiqierienoes  widi  die  Geometry  tutor  was  so  carefiilly  contndled  by  die  immediate  feecfoack 
piiiid(des  of  its  operations,  die  tutor  guaranteed  that  students  erqierienoes  were  much  more 
uniform  and  similar  dum  was  the  case  for  nmmaldassrooms.  As  a  result,  students  could  mote 
easily  dune  mqietiences  and  make  use  of  one  another’s  experiermes  and  proUem  solving 
strat^ies.  Ihe  practical  result  was  a  great  deal  of  cooperative  problem  solving. 

Ccmdiisioiw  fiom  the  Six  Evaluation  Studies 

Hiese  evaluation  results  all  qipearvery  positive  r^atditig  the  efficacy  of  ITS;  howev», 
diete  is  dways  a  setectkm  Mas  invdved  with  the  puUication  of  unambiguous  evidence  of 
successful  instructional  interventions.  We  ate  familiar  with  other  (urqwblishedltulor-evduation 
stwfies  that  were  conducted  but  woe  "fitihires."  However,  the  genetd  positive  trend  is  viewed  as 
encouraging,  especially  given  die  enormous  difiennices  amtxig  die  six  mtors  in  design  structure 
as  well  as  evduation  methods.  Ihefindii^  indicate  that  these  systems  do  accelerate  learning 


widino  d^radatkm  in  find  outcome. 

Obvioudy,  princiided  qiproaches  to  both  the  design  and  evatuatkm  of  ITS  are  badly 
needed  before  we  can  definitively  judge  the  merits  of  these  systems.  Some  ptindided  qiproadies 
areb^inningtoenierg. .  Fbr  examine,  KylloienaidSliute  (1989)  outlined  a  taxmcnny  of 
learning  skflls  that  has  implications  for  die  systemdicdnfgR  of  ITS.  Theyhypothesizedamulti- 
drmenskmd  interaction  predicting  learning  outcome  as  a  function  of;  0^  of 
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leamiiig^iistiuctkmal  aMionment,  type  of  knowledge/ddll  being  instnicted,  subject  matter,  and 
diancteii8tic8oftiieleanier(e.g.,  aptitude,  leamiiig  style).  a  few  modificatioas  to  this 
tnonomy,  R^an  and  odleagues  at  the  Annstnmg  Laboratoiy  are  currently  trying  to  fill  in  the 
cells  in  die  matrix  duough  systematic,  empirical  studies  designed  to  assess  perfonnance  acn»s  a 
range  of  these  afiMcmendoneddimensimis.  Iheir  goal  is  to  mqiinstrucdcmal  and  knowledge- 
type  variables  to  learning. 

In  terms  of  systematic  qtproadies  to  evaluating  ITS,  anite  and  R^ian  (1993)  suggested 
seven  8tq»  for  ITS  evaluation:  (1)  Delineate  goals  of  the  mtor,  (2)  Define  goals  of  the 
evahiadon  study,  (3)  Select  the  ^)piopiiate  design  to  meet  defined  goals,  (4)  Instantiate  the 
design  with  appropriate  measures,  number  and  type  of  subjects,  and  control  groups,  (S)Midre 
careful  logistical  preparations  for  conducting  the  Audy,  (6)  Pilot  test  tutor  and  otho'  aqrects  of 
the  study,  and  (7)  Flan  primary  data  analysis  concurrent  widiplanniQg  the  study.  These 
ptindides  mi^  also  be  enqdoyed  as  a  framewmk  for  organizing,  discussing,  and  comparing  ITS 
evaluation  studies. 

FUTURE  ITS  RESEARCH  AND  DEVELOPMENT 
What  is  possMefar  the  future  includes  ample  computing  resources  fqr  every  student 
...  U^tping  electronically  many  resources  outside  the  classroom.  It  includes  the  idea  qf 
a  personal  factotum  dtat  could  serve  as  a  knowledgeable  intermediary ...  to  bridge  the 
gap  between  the  classroom  and  the  external  world ...  Virtual  field  trips  linking 
libraries  and  museums  will  have  their  holdings  available  in  electronic  (or  photonic) 
form ...”  (Nickerson.  1988,  p.  312). 

We've  seen  where  ITS  research  and  de^opmettt  has  been,  and  we’ve  discussed  a  few  of 
tile  systons  that  have  been  evaluated  in  controlled  studies.  Well  now  examine  some  of  tiie 
oonceividtie  futures  for  these  systems.  Given  tiie  diversity  of  researchers  in  tiie  area,  and  the 
great  (fifferences  among  learners,  tiiere  will  be,  in  reality,  many  diffinent  streams  of  leseandi  00* 
occurring  and  tiie  most  likely  future  is  probably  a  composite  of  tiiem  an. 

PUtare  t !  Immeffgive  Leaminy  Envimnmentg  Evolve  from  ITS 

Mden  (age  11)  walks  into  his  culricle  at  school  and  excitedly  puts  on  his  VR  body-suit. 
Today's  itinerary  (jointly  produced  by  Alden  and  his  main  teacher)  is  teeming  with  new  learning 
adventures.  After  taking  a  Dramamine,  he  boards  a  boat  heading  up  the  Nile.  This  trip  (and  his 
on-line  tour  guide)  wiil  he^  him  learn  about  East  Afriat's  geography,  fiora,  and  fauna  as  he 
cruises,  observes,  hears,  and  smells  things  along  the  world’s  longest  river.  When  the  trip 
concludes,  he  plans  on  visiting  Olduvai  Gorge  for  sane  archeological  excavations  (tfier  all,  he’s 
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atready  in  Africa).  SpecifrcaUy.  AUen  will  get  a  chance  to  help  dig  out  some  early  human 
remains.  Then,  for  charge  of  pace,  Atden  and  his  VR  pal  Rafael,  who  lives  in  Mexico  City,  will 
meet  in  a  happenin' ^pace  station  they  programmed  together.  They  are  learning  each  others' 
language  and  culture-Rtfdel  steaks  English  and  helps  AUen  learn  Spanish,  while  AUen  speaks 
Spanish  and  assists  Rafael  with  Ms  English.  Following  a  real  lunch  (not  a  virtual  one,  as  all 
Ms  learning  makes  one  hungry),  AUen  concludes  his  day  on  an  artistic  note.  He's  creating  a 
VR  masterpiece  representing  his  interpretation  of  die  classical  score,  "The  Wall"  by  the  noted 
composer  Roger  Waters,  designing  virtual  sadptures,  their  choreography,  and  musical 
arrangement. 

lliis  imsgined  futuie  using  iDunersive  Learning  EnvironmeiMs  can  attain  its  instn^^ 
gods  as  fidlows.  As  a  glnstpiesence,  the  tutor  in  tbese  new  systems  can  interact  witfi  a  student 
thiou^  digital  speech,  duough  text  that  floats  in  the  air.  or  tfarou^  replays.  Asanembodied 
presmce,  the  tutor  can  vary  in  reality  from  a  stick  figure  to  a  realistic  mannequin,  with  £Kid 
etqxessitms  and  voice.  Ihe  possibilities  for  realistic  guidanoe  that  is  as  bdievaUe  and  as  finceful 
as  a  real  tutor  may  be  quite  difficult  to  addeve,  but  it  can  be  dramatic  in  implications.  The 
bdievability  of  these  new  systems  hinges  on  die  quality  of  the  immersive  eiqterience  titey 
provide.  The  differences  between  an  Immerrive  Learning  Environment  and  its  2D  simulation 
counterpart  depends  upon  the  results  of  immersiop  and  in  the  different  ways  ttiat  students  can 
imeiact  witti  the  woild.  Instead  of  moving  a  mouse  or  a  joystick,  kanen  can  move  their  own 
hands  to  pick  something  iq).  Although  they  mi^  not  feel  the  object  accurately,  there  are  enough 
cues  to  provide  tiKseiuotioR  of  piddng  tilings  iq>.  RrsLtiiey  see  it  happening,  and  vision 
dearly  dominates  other  senses  to  provide  a  compdling  illusion.  Contad  and  force  can  be 
provided  realistically  with  expendve  fmce-fiBedbadt  devices,  or  suggestivdy  witii  sounds,  sudi 
as  a  pfiig  tiiat  denotes  odlision  or  toudiing. 

VR  also  opens  tile  opportiniity  for  provkhr^  handicapped  or  disaUed  pec^  an 
experience  of  unfettered  motion;  or  new  imerfaces  to  comrd  the  world  witii  minimal  movements. 
It  can  make  invisibie  fnces  like  gravity  and  air  pressure  visible  and  hence,  more  oonqxehendUe 
tostudents.  For  instance,  Minstrell  (1988)  poimed  out  tiiathi^schod  students  go  through  a 
period  of  misconceptions  (hiring  which  tiiey  confuse  graviQr  and  air  pressure;  so  tiiat  vdien  air  is 
pumped  out  of  a  bdl  jar,  objects  inside  it  are  expected  to  become  lighrer  or  even  float  VRoffers 
an  opporturtity  fordoing  a  set  of  expeiimenis  in  wUch  the  forces  of  gravity  and  air  pressure 
could  be  made  vidUednoughgnqMc  icons,  such  as  cttiored  arrows  or  textures.  Astiiegasis 
removed  from  a  bdl  jar,  it  could  be  visible  as  a  cttiored  gas  flowing  out  Students  could  actually 
reach  into  tile  ben  jar  and  manipulate  the  objects  as  the  gas  is  ranoved.  They  could  even  adopt 
tile  pctint  of  view,  or  flame  of  reference,  of  an  object  inside  tiie  beU  jar,  and  eiqieiience  die 
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change  to  fbroesdiieGtly.  MaUng  these  forces  visible  in  a  nniltitiide  of  lifelike  and  bdievaUe 
envinxinenls  niiqr  have  pndbimd  effrcts  on  children's  undentanding  of  sdence. 

It  tdKMild  be  noted  that  the  same  pioblenis  tint  plague  ITS  are  relevant  to  VR.  Hiatis. 
die  emphasis  needs  to  pethsps  shift  away  from  omnfooiBiit  VR  systems,  towaid  t  ctdlectkm  of 
specific  mini-syslenis  and  goals  (e.g.  teach  die  knouiet^e  of  X,  the  ddll  of  Y,  and  piovide  die 
Unesthmic  feedback  ftnr 


Fteme  2:  TYadMonal  ITS  Disanpear.  Snedfic  Cognitive  Tnnls  Dnminate 

Whitney  (age  14)  arrives  in  her  classroom  and  takes  a  seat  at  her  learning  station,  a 
large  comfortable  desk  wiA  an  embedded  computer.  The  touch-screen  is  divided  into  many 
deferent  areas  that  have  distina  functions  (e.g.,  graplucs,  spreadsheet,  sound  analyzer,  dozens 
cf  databases).  From  the  from  of  the  class,  a  visiting  detective  (serving  as  the  day’s  teacher) 
accesses  the  international  police  database  (IPD)  and  obtains  details  surrountting  a  grisly 
n  ^  that  happened  the  prevUmmondi  in  a  small  Italian  city.  She  electronically  transmits  all 
oj  information  to  tite  students,  which  includes  electronic  photogroftiu  of  the  physical 
evidence  (e.g.,  the  body  and  tite  wee^n),  psychological  profiles  of  the  victim  and  11  aupects, 
recorded  interviews,  alibis  and  mtdlves,  phone  logs,andsoon.  The  students  have  to  engage  in  a 
varietyof  coordinated  cognitive  activities  to  solve  the  murder  mystery.  Whitney  first  brings  tqt 
die  psychological  profile  of  the  dead  man.  After  reading  tiieftle,  she  notes  in  her  electnmic 


scratchpadthatthevictimhadahistory  of  drug  abuse  and  depression.  On  another  part  of  her 
25”  screen,  she  accesses  a  3D  photo  of  die  victim,  zooms-in  on  Ids  arms,  and  sees  eddence  of 
two  recent  intraventm  ihicctions.  The  paduthgy  report  fivmdte  coroner’s  office  concluded  dutt 
die  dcdm  ddedfrom  a  gunshot  wound  to  his  heart,  but  traces  of  a  narcotic  substance  were  also 
found  in  his  body.  Playing  die  interview  tapes  on  her  "stress  analyzer,"  Whitney  discovers  dust 
two  of  the  subsets  are  clearly  tfing.  Throughout  die  day,  puzzle  pieces  slowly  come  together, 
die  dmective-teadier  offers  a  few  suggestions,  and  finally,  Whitney  figures  out  whodunit  (with 
S3  probaltiUty  of  accuracy). 

hi  diis  vision  of  die  fiituie,  "(Hnn^xHBtt"  inteUigett  tutoring  systmns  have  been  iqdaoed 
by  collections  of  specialized  edocatknal  or  cognitive  tools-tecfancdogical  devices  diat  heip 
people  to  pefftnmcc^nitive  tasks  (i,e.,lie4>  diem  know,  think,  or  learn).  Forexample, 
shnulslarB,  smait  qxeadriieets,  and  extensive  dmabases  are  cognidve  uxds  available  within 
dassrooms.  ^iprenticeship  training  is  enviskmed  as  the  main  source  of  impaiting  drill,  in 
oonfuocdoQ  with  the  siqipfemental  dmulattn  nd  associated  tools  for  die  ipprendoe  to  emidoy 
duffa^  leaniing.  Hk  training  situations  idate  to  real-worid  evems,  dus  placing  teaming  widiin  a 
meaningful  context 
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OiM  iMOon  thit  ITS  may  dis^ipear  in  the  fotoie  is  ttutt,  whik  mjny  reseaichen  a 
ttMt  tattell^enoe  in  n  rrs  is  directly  a  function  of  Ae  pitsence  of  •  student  model,  the  studem 
nmdd  auy,  in  iKt.  be  dm  wrong  fnmewoik  around  wMch  to  build  good  learning  niadiines. 
Deny  and  Li^oie  (1993)  presented  six  reasons  why  the  student  moddingpaiwBgm  is 
pioblanatic:  ( 1 )  In  complex  domains,  the  student  modd  camot  specify  all  possible  stdution 
paths.  (2)Oneaumotdeanniiiieorinduoe  all  possible  "buggy”  behaviors,  (3)  "Canned"  text  is 
antithetical  to  principles  of  tutmial  dialog,  (4)  Reflection  and  diagnosis  should  be  peifonned  by 
the  not  die  tutiv.  (5)  hnplenienting  the  student  modeling  approach  is  veiy  difficult, 
tedmically,  and  (6)  Moddl>tracing  is  only  tgjplichbte  to  procedural  teaming,  but  die  focus  dKNild 
be  on  ciRical  ddnking  and  problem  soivhig. 

A  econd  factor  that  coold  contribute  to  the  decline  of  ITS  is  that  the  tenn  "intelligent 
tmoifaig  qnlem"  is  associated  widi  philosophical  issues  idadng  to  the  nature  of  intdligence. 
Many  people  assodale  intelligenoe  with  awareness  and,  since  no  AI  system  could  be  said  to  have 
achieved  awareness,  diese  people  would  not  grant  that  any  rrs  had  ever  been  devdoped. 
Neveithdess.  domns  "inlelligent"  tutoring  systems  have  been  routinely  rqxxted  in  the 
literature,  and  even  more  discussed  at  conferences.  So,  die  name  (and  hence,  die  vdKde 
enteiprise)  may  be  hugipropriate  or  mistearBng.  Sinqily  put,  ITS  may  promiae  too  much,  ddiva 
too  Ittde,  and  constitute  too  restricdve  a  constnict  Ougeity  (1993)  summed  it  iq;>  best  as. 

There  is  a  sense  in  which  Ae  goals  ofmuUtkmal  hitelligent  tutoring  systems  are 
boA  too  ambitious  and  too  narrow.  Most  tradiHonal  ITS...  are  designed  to  provide 
tutoring  A  a  stand^done  settAg...  This  ambitious  goal  requires  Aat  the  ITS  hemdle 
all  a^tects  of  Ae  very  d^fladt  task  of  tutoring,  includAg  expert  problem  solving, 
student  dAgnosis,  tailoring  insuvction  to  changAg  stueknt  needs,  and  providing  an 
instructional  envirtmment...  On  the  other  hand,  Ae  goal  of  developing  very 
intelligent  stand-alone  ITS  is  narrow  A  Ae  sense  Aat  it  limits  our  conception  of  how 
intelligence  can  be  Acorporated  into  conqtuter-based  training  and  education  (p.  3). 

As  a  parallel,  consider  what  hqipened  in  die  field  of  robodcs.  Rrst-generadon  robots 
were  oonstracted  out  (rf  pure  research  curiosiQr.  Then,  after  die  initial  flurry  of  excitement  in  the 
1960b  and  eaily  19708  tfed  down,  emphads  shifted  fiom  building  single-systnn  robots,  to  mcne 
»niphMi«  on  twiiding  cnmpfvyit  partii  Thls  tTsde-off  wss  due  to  the  problems  associated  wlfii 
designing  a  system  that  has  genendiwipoae  proUem-solving  skills  vosus  one  widi  more  focused 
expertise.  The  next  generation  of  robots,  arising  form  the  work  being  done  on  die  indivkhid 
parts,  may  reacdve  dds  conflict  by  becoming  an  expert  in  a  given  donudn,  but  also  possessii^  a 
wide  repertoire  of  goanal  problem-solving  skills.  The  same  applies  for  ITS.  Radwrdum 
attenqitiog  to  boUd  an  onuiipotent  totin’,  a  mme  fruitful  qiproach  might  be  to  oeate  a  cohoem 
ooitection  of  computerized  ttxds  (i.e.,  a  divide-and-conquer  strata). 
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CurHs  (age  9)  rolls  out  of  bed.  greets  his  parents  (already  at  work  in  their  cubicles),  eats 
breakfast,  glances  at  the  sleet  Jailing  outside,  then  ambles  over  to  his  computer  for  his  morning 
curriculum.  Curtis  “goes  to  school"  in  his  home.  When  he  logs  onto  the  Public  School  System, 
he  first  checks  his  mail,  then  receives  a  menu  of  options  for  the  morning's  lining  project: 
Would  he  like  to  learn  about  Tyrannosaurus  Rex.  the  politics  leading  up  to  World  War  II,  or 
what  caused  the  Califomia  earthquake  of 1994?  All  he  has  to  do  is  tap  into  the  appropriate 
database,  travel  to  the  correct  geographical  region  and  time  period,  and  interact  widi  these 
reactive  environments  through  the  multi-media  systems.  The  reqtective  databases  all  include 
on-line  hosts  to  narrate  events  and  answer  questions,  movies  to  depict  a  range  of  relevant  topics 
(from  mundane  to  crucial),  and  simulators  to  allow  Curtis  to  experiment  within  the  d^erent 
worlds.  After  choosing  T.  Rex  as  his  learning  project,  dte  host  narrates  some  basic  declarative 
information  (e.g..  when  they  existed  and  for  how  long,  size  of  the  dinosaur,  diet,  mating  htdrits, 
other  co-existing  plants  and  animals)  then  Curtis  uses  the  Emulator  to  martipulate  geological 
events  to  see  their  ramifications  on  tite  dinoseutr.  The  first  Mng  he  does  is  to  reverse  the 
advandng  ice  age  (introducing  a  global  wamtir^  trend  in  Its  place),  and  then  sees  its 
implications  on  not  only  the  survival  of  the  lizard  king,  but  also  on  the  evolution  of  other  plants 
and  animals  on  the  planet.  Periodkatty,  the  host  asks  for  some  predictions,  Curtis  responds, 
and  receives  feedback  from  the  host.  On  ocatsion,  other  students  in  the  same  module 
communicate  their  finding*  and  questions  to  him  over  the  network  lines. 

As  can  be  seen,  ttdsfutmc  is  attractive  for  a  lot  of  fcasnis.  distance  teaming,  me 
can  allow  leameis  to  stay  at  borne  or  some  other  convenient  learning  location  (saving  time  and 
transportatkm  costs),  and  connea  to  a  rich  netwoik  of  infonnation  and  training  software, 
available  across  an  infonnatim  siq)er-highway.  To  achieve  fois  future,  expert  systems-qNuming 
a  huge  anay  of  possible  domafos-are  needed  that  piesem  comptehenrive  infonnatim,  as  well  as 
provide  tihought-piovoking  questions,  and  lespcmd  to  student-directed  queries.  Tbenetworic 
dmuld  also  allocate  nodes  to  which  one's  peers  can  be  connected,  tfius  providing  for 
collabcvative  teaming  opportunities.  Notice  foat  this  distance-learning  future  is  not  limited  to 
wyt»fgitig  dedantive  knowledge  fiom  databases.  Radier,  software  (e.g.,  simulators)  should  dso 
be  accessiUe  to  pracdce  skill  in  any  qiecific  domain. 

In  this  ftiture,  it  is  possiUe  to  quiddy  access  on-line,  digital-rich  lOtraries  with  virtually 
limiflftM  t»»nlnii^rfafahn«i^  fhr  niir  peiawial  learning  i^easure.  And  while  the  educatiCHial  horizCHl 
win  invaiitddy  indude  VR  technology  as  an  important  instracticmal  medium  (see  Fbture  1),  it 
win  be  just  one  of  many  media. 
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Hnaity,  to  attain  this  future  and  the  metaphor  and  (Homise  of  the  library  as  a  knowledge 
space  (i.e.,  the  epitome  of  Carbonell’s  dream  and  the  hypertext  vision),  we  must  first  maka  a 
fundamental  change  how  we  think  about  education.  Our  narrow  oonceptimi  of  education  (e.g., 
"aclKxd”),  only  retevant  for  those  between  the  ages  five  and  eightemi,  is  no  kN^r  ^ipropri^. 
Education  Aould  be  for  everyone,  all  ages,  and  avaiiaUe  in  all  places. 

Future  A-  I  ^tniny  ia  Oil  Cnn«hnr»rtv>>  I  .Mmme  i«  In 

Sierra,  Nicole,  Fernando,  Sasha,  Kevin,  and  Uri  comprise  "Team  3. "  They  are  between 
the  ages  of  18-22  (college  sophomores).  In  dteir  sociology  class,  there  are  two  professors  and 
five  teams,  each  team  r^ecdng  an  t^timal  mixture  of  aptitude,  gender,  learning  styles, 
personality  types,  and  eduiic  backgrounds.  They  are  all  geared-up  for  their  on-line  VR  lesson  on 
"racial prejudice."  The  six  students  are  tran^rted  to  Birmingham,  Alabama  on  a  hot  August 
day  in  1951.  In  reality,  only  Sasha  and  Kevin  are  African-American,  but  in  Ms  lesson,  all  six 
kids  are  transformed  into  "Negroes"  (as  they're  cedled  in  1951).  The  lesson  requires  diem  to  take 
a  city  luis  to  a  "Whites  Only"  park  that  has  a  nice  public  swimming  pool,  try  to  swim  in  the  pool, 
dten  go  hone  to  dteir  impoverished  residences  on  the  outsorts  of  town.  Problems  arise 
Immediately  in  this  compelUng  Emulation  when  dtey  board  the  bus.  Automatically,  they  all  sit 
down  in  the  front  seats:  after  all,  there  are  only  four  odier  riders  on  die  bus,  sitting  in  the  middle 
section.  The  white  bus  driver  rudely  informs  them  to  "move  to  the  back"  whereupon  Sierra 
(T earn  3's  outspoken  leader)  politely  atda  "why?"  When  she  gets  slapped  for  her  impudence, 
Nicole  starts  to  cry.  But  Sierra  persists.  Then  the  bus  driver  utters  stme  very  ugly  sentiments 
abmtt  them  all,  based  solely  on  dteir  skin  color.  They  see  by  his  reddemng  face  and  posture  that 
he's  about  to  strike  out  again,  so  they  collective^  decide  to  move  quickly  to  the  back  of  the  bus. 
During  the  ride  to  the  park,  they  discuss  their  experiences  (what  they  feel,  what  they  could  have 
done  d^erently,  what  caused  diis  state  of  affairs,  ete.).  Sasha  and  Kevin  contribute  valuable 
information  to  die  discussion  fivm  personal  tales  related  to  than  by  their  grandparents  and 
gyeat-grane^Htrents.  Finally  they  arrive  at  the  park,  and  dungs  really  go  downhill  fr<m  there. 
They're  not  allowed  to  enta  the  park  or  swim  in  the  pool,  they're  called  "dirty"  and  worse,  and 
the  sinudadon  makes  them  all  painfully  aware  of  racial  prejudice.  Afterward,  Team  3  reviews 
and  discusses  all  cf  the  events,  and  their  professors  provide  irformation,  as  needed,  about  the 
historical  roots  of  racial  prejudice  leading  up  to  die  situation  they  encountered  in  dieir  lesson. 

Hie  motivating  force  driving  diis  future  is  the  belief  that  collaborative  learning  is 
superior  to  individualized  teaming.  That  is,  learning  may  be  invaluaUy  enhanced  from 
conversations  with  ttiose  udio  have  differing  of^mis,  backgrounds,  or  skills,  know  more  about 
some  topic,  or  who  cm  ask  perceptive,  thought-provoking  questimis.  Basic  research  is  being 
conducted  in  cognitive  and  social  psychology  that  seeks  answers  to  questimis  pertaiidng  to  the 
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ofitimalooiiipositioiis  of  learner  gioiq».  Some  of  these  research  questkms  include:  Is  it  better  to 
mix  genders,  or  have  m(»e  homogeneous  gfouiriiigs?  When  estatdidiing  groups  based  on 
aptitude  levels,  is  it  better  to  match  highs  with  highs,  or  a  high  with  a  low?  What  are  the  optimal 
coorcMnations  of  affective  characteristics  (e.g.,  passive  with  gregarious)?  And  what  other 
cognitive/sodal  considnations  should  be  made  (e.g..  letting  individuals  self-sdect  their  group  vs. 
being  assigned)?  According  to  Resnidr  and  Johnsrai  (1988),  sociological  studies  show  that  most 
peofde  prefer  personal  sources  of  information,  and  cranputers  can  enlkjice  sireb  cmnmunicatxMis. 

Technology  is  evolving  to  the  point  where  cranputer  systems  can  routinely  anttain 
leaming  enviramients  that  support  a  high  level  of  social  interaction.  This  important  technology 
focilitates  effective  teaming,  especially  within  the  classroom.  Ibe  atmospheres  in  tiie  classrooms 
contairting  tiie  cmmected  computerized  environment  are  boisterously  amtrolled,  similar  to  what 
Fenrzdg  (1988)  found  in  a  collaborative  mathematics  course  that  was  "..more  like  a  beehive 
than  a  math  class."  Q>.  1 17).  These  collaborative  clamxmis  can  even  support  rretworked  VR, 
which  means  that  students,  trainees,  and  experts  can  interact  between  schools  and  mnote  sites, 
and  tiiat  trainees  and  instructors  can  share  tiie  same  experience.  Learners  can  wmk 
collaboratively  (Ml  tile  same  project  On  the  other  hand,  differem  students  can  work  (mi  the  same 
project  at  the  same  time,  witimut  awareness  of  eardi  (Mher's  presence,  but  with  s(Mne  invisiUe 
instructor  lurking  over  their  shoulders.  The  number  ofcomUnations  are  staggering,  and  their 
leaming/training  potential  is  unknowrL 

The  ottier  person  in  the  networired  world  could  also  be  an  autonmnous  agent  or  cyborg, 
part  real  and  part  synttietic.  This  idea  raises  a  whole  new  set  of  posstixlities  for  a  computer 
coach,  eiqtianaticMis,  and  guidance.  "Social  interface  agents"  (Thotissnt  1993)  have  inogressed 
steadfly  as  informatitMi  dxMit  how  to  ditect  gaze,  when  to  use  paraverbals  (hmmm,  uh ...)  and 
when  to  take  toms  in  a  dialogue,  all  become  bettm’ understood.  Iminovorrents  in  modding 
human  actions  and  planning  (e.g.,  Badter,  Phillfos,  and  Webber,  1992),  including  natural 
language  interaction,  will  soon  lead  to  the  devdopmoit  of  virtud  agents  tiiat  can  coach  and  guide 
learners'  actions  within  carefully  {tianned  leaming  activities.  Some  of  tiiese  interactions  are 
already  available  in  a  tmrt  form  (Curtis  and  Nichols,  1993).  Ihese  virtual  agents  focus  (m 
students'  errors  by  offering  erqierts'  stories  (Kedar,  Baudin,  Bimbaom,  Osgood,  and  Bareiss, 
1993).  Networited  digital  places,  such  as  digital  libraries,  demand  new  tedmiques  for  navigatir^ 
through  diese  comjtiex  graces  without  getting  lost  Issues  of  bow  to  maintain  a  smse  of  locaticm 
(Benedikt,  1991)  and  how  to  best  use  these  aivirmiments  to  support  memory  witii  the  method  of 
lod  (Neisser,  1987)  need  more  researdi. 

As  shown  in  the  above  illustraticm,  VR  provides  a  new  saliency  cm  tiie  notion  that  scnne 
tfaiiigs  (su(di  as  race  and  gender)  are  constructed,  and  that  we  can  become  what  we  play,  argue 
about,  and  build.  For  instance,  text-based  VR  already  invites  the  participation  of  wcxnen  and 
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glMs  in  social  interactions  in  ways  that  adventure  gvnes  like  (hmgecnis  and  dragons  did  not 
Collide,  1993).  Tuikle  points  out  ttiat  MUDs  (i.e..  multi-user  dungeons)  are  easily  used  for 
genderswqii^.  When  goxler  roles  are  switched,  sexist  expectations  and  overt  demands  that 
migitt  be  ignored  in  daily  life  become  highly  visible  and  reactive,  aid  diey  are  openly  discussed. 
Hie  MUD  then  becomes  an  evocative  tfeject  for  a  richer  understanding  not  tmly  of  sexual 
harassment,  but  of  the  social  amstiuction  of  ^nder. 


^  1 1 1 H  I  i  i;  • ;  1 1?:  J 1 1  *  1 1 1  *  ^  iTTf  nnTMlfTTTiT 


Wesley  (age  10)  arrives  at  the  math  lab  wlure  he  sits  in  front  of  a  computer  that  is  going 
to  he^t  him  learn  to  solve  algebra  word  proMems  better.  Today's  focus  is  on  those  troublesome 
distance-rate-time  problems.  After  stating  his  name,  the  computer  accesses  Wesley's  records, 
flttggutg  his  salient  strengths  and  weaknesses  (i.e.,  not  only  his  higher-level  aptitudes,  but  also 
the  low  level  productions  that  he's  acquired  and  not  yet  acquired).  Beginning  with  a  review  of 
concepts  and  skills  that  he  learned  the  day  before,  die  ITS  generates  a  proUem  which  is  Just  a 
litde  Irit  out  of  his  grasp.  The  ITS  then  works  out  the  correa  solution  to  the  problem,  along  with 
an  alternative  solution  that  Wesley  is  very  likely  to  come  tq>  with  based  on  its  student  model  of 
him.  Infuct,  he  solves  the  problem  exactly  like  the  tutor  pralictai.  As  part  of  its  student  model 
of  him,  the  ITS  "knows"  to  instruct  Wesley  with  an  emphasis  on  a  graphical  representation  of  the 
proMem  to  chilly  the  dhscrepancy  between  the  correct  and  incorrect  solutions  and  facilitate  the 
formation  of  afunctional  mental  model.  Thus,  die  tutor  presents  two  animated  trains  appearing 
on  opposite  sides  of  the  screen  that  converge  at  a  point  almost  in  the  middle  of  the  screen.  They 
travel  at  threat  rates  of  speed.  The  problem  statement  stays  up  at  the  top  of  the  screen,  and 
the  tutor  points  out,  as  it  periodically  pauses  die  simulation,  what  elements  should  be  attended  to 
andndien,  Wesley  states  that  he  understands  die  mtqtping  between  the  explicated  mental  model, 
the  a/^ropriate  equation,  and  the  relevant  parts  of  the  word  problem.  So  the  ITS  presents  an 
isomorphic  word  problem.  This  time  he  solves  it  correctly,  without  any  supplemental  graphics. 
Wesley  exercises  an  option  to  play  around  with  some  trains,  missiles  and  boats  on  his  own  for  a 
while  to  test  Ms  emerging  understanding.  He  viem  Ms  "score"  of  curricular  elements  acquired, 
and  seems  a  little  frustrated  about  his  progress,  but  the  ITS  reassures  him  that  he  is  proceeding 
at  a  reasonable  rate.  Instruction  and  learrnngcominue. 

For  ITS  to  evolve  to  ttie  point  seen  in  the  above  scenario,  more  amtrolled  research  must 
be  cmducted  in  three  areas  of  inteUigence:  the  dmnain  mqiert,  the  studmit  model,  and  the  tutor. 
Rnt,  die  subject  matter  must  be  understood  the  computer  well  enough  for  the  mnbedded 
expert  to  drew  inferences  or  solve  proUems  in  the  domain.  Next,  the  system  must  be  aUe  to 
deduce  a  leainer*s  approximation  of  that  knowledge.  Fuially,  the  tutorial  strstegy  must  be 
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imelligent  to  the  p(^  where  an  on-line  tutor  can  im|danent  strategies  to  reduce  the  differences 
between  the  ejqwit  and  student  perfonnance  (Bums  &  Capps.  1988). 

Solutitms  to  probtems  involving  difficult  AI.  psychology,  and  pedagogy  will  emeige 
fiom  research  endeavoR  that  yield  infonnation  about  effective  and  efficiem  ways  to  (a)  represent, 
utilize,  and  communicate  domain  knowlec^e.  (b)  reimsent  an  individual's  evolving  knowledge 
state  (for  botti  declarative  knowledge  and  inocedural  skill),  and  (c)  instruct  the  material  most 
effectively  for  a  particular  learner.  Some  specific  research  questions  include:  How  can 
cmuputeR  better  understand  natural  language  (it^xit  as  well  as  output)?  What  kinds  of  inference 
mechanisms  can  optimally  model  students'  knowledge  status?  How  can  computCR  be 
programmed  to  understand  "semilogical"  reastming  (including  intuiti(Mis,  pet  theories,  prior 
experiences)?  What  are  the  specific  characteristics  of  leameR  who  perform  better  in  certain  types 
of  teaming  environmertts  and  not  in  ofoere?  Are  certain  dcmiains  better  suited  for  specific 
instiucticmal  methods?  When  should  feedback  be  provided,  what  should  it  say,  and  how  best 
riioukl  it  be  presented?  How  much  learner  ccxitrol  riiould  be  allowed? 

Some  addititmal  limitations  of  current  ITS  have  alrearty  been  mentimied  (e.g..  student 
modds  cannot  speciiy  all  possiMe  solution  paths  in  com{dex  domains,  model-tracing  is  rally 
suitable  for  procedural  learning).  Otk  possible  solution  would  be  to  use  a-kind  of  model-tracing 
qiproadi  for  instructing  well-defined  procedural  skills,  using  an  underlying  expert  and  student 
model  foat  are  {ximarily  rule-based.  And  for  instructing  dedarative  information  or  cranplex.  iU- 
structured  domaimi,  the  ITS  may  include  a  knowledge  base  that  is  a  semantic  nd  wifii  extensive 
indexing  (like  CBR). 

Whatever  future  ultimately  evolves  firan  ITS,  the  fields  of  AI,  educatirai,  and 
psydiolt^  have  pioStrd  enormously  from  the  craitributirais  made  in  the  ITS  arena.  Learning 
ttieories  have  been  tested;  individoal  differences  issues  have  been  validated  against  complex, 
leal-wiHld  learning  tasks  (e.g..  ITS,  in  craitrast  to  artificial  laboratory  tasks);  AI  programming 
techniques  have  been  refined;  different  instructkmal  t^rproaches  have  berai  compared,  controlled 
studies  conducted  of  iqjtitude-treaimem  interactirais,  and  so  forfii.  So,  in  terms  of  research 
vetiictes,  ITS  are  greatly  underestimated.  But  for  purposes  of  education,  their  time  may  be 
limited;  maybe  not 


CONCLUSIONS 

Before  the  computer  age,  the  inevailing  instructional  qrproach  was  sufficioit  (e.g.,  one 
teacher  transmitting  infotmatirai  to  about  30  students),  but  we  now  reside  in  a  computerized 
world,  tnitisi  implementations  of  CAI  mirrored  this  pedantic  approach,  and  to  some  extent  so 
does  file  cunenfiy  popular  model-tracing  tqrproadi  in  sqrhisticatedITS.  Do  we  need  to  change 
our  etbicational  philoso|Mes  or  systems? 
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We  have  most  of  the  components  oecessaty  to  advance  educational  refonn.  Not  only  is 
there  great  need  for  change,  but  also  there  are  powerful,  affoidaMe  technologies  availaUe  to 
support  It  Missing  are  definitive  answers  to  the  psydiological  controversies  cited  earlier.  Basic 
researdh  is  actively  being  pursued  to  resolve  these  issues.  For  examine,  studies  are  beginning  to 
consistently  find  that  higher-order  drinking  skills  are  not  acquired  through  didactic  apfmraches 
(i.e.,  strain  amveyance  of  facts),  but  radter,  through  learners'  active  involvement  with  the 
subject  matter.  Hiis  "constructivist"  view  of  learning  allows  students  to  achieve  intellectual 
accmniriidiments  not  possible  under  more  traditional  pedagogical  apfHoaches  (Collins,  Brown  & 
Newman,  1989;  Resnick,  1987). 

The  taUe,  below,  contrasts  old  versus  new  approaches  to  instiuctimi  (from  Means, 
Blando,  Olstm,  Middleton,  Morocco,  Remz  &  Zorfass,  1993): 


TaUe  2:  (Md  versus  New  A 

pproaches  to  InsttuctitMi 

Old 

NEW 

Teacher-directed  activities 

Stiident-ditected  exUorations 

Didactic  teadiins 

lmaw:tive  modes  of  instractimi 

ExtOKted.  multidisciUinarv  instraction 

Individual  work 

Collaborative  work 

Teacher  as  knowledge  dispenser 

Teacher  as  facilitator 

Ability  groupings 

H^erogeneous  groumngs 

Assessment  of  factual  knowledge  and 

discrete  skills 

Perfonnance-based  assessment 

This  taUe  provides  a  clear  direction  for  ITS  research  and  imidenrentatitm.  That  is,  to  get 
from  "old"  to  "new,"  we  need  to  open  rqr  tearning  oivironments  drat  promote  increased  learner 
initiative  and  betwear-learner  collabotatiotL  We  should  assess  learning  as  it  transfers  to 
authentic  tasks,  not  standardized  tests,  and  atterrqrt  to  estatdidi  connections  across  various  fidds 
so  topics  are  not  learned  in  isolation  of  one  anotiier.  As  technologies  emerge  and  advance,  we 
can  fit  diem  into  this  fi«mework.  Furthermore,  additkmal  research  is  needed  to  validate  the 
goodness  of  the  new  over  the  old  qiproach  to  teaching-teaming. 

Look  around  you.  Computer  technolr^es  have  dramatically  transformed  the  workplace, 
cmnmunicadons  and  cmnmercial  activities,  as  well  as  the  entire  business  community.  But 
education  rmnains  srarus  quo.  We  need  to  harness  the  computer's  potential  and  find  ways  to 
emidoy  it  in  prmnoting  educatimial  change.  Are  current  and  prevalent  ITS  adequate  for  our 
puiposes-^w  and  in  the  21st  century  (just  right  around  the  comer)?  We  believe  that,  as 
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cunenfly  implemenied.  tticse  systnns  may  have  asymptoted  in  utility.  A  philosophical  shift  has 
been  suggested  in  this  cbqiier,  away  from  stand-alone  Instructional  devices  and  toward  using 
tools  to  aid  in  the  more  collaborative  learning  (Hocess.  There  are  actually  very  few  ITS  in  place 
in  schools,  yet  they  exist  in  abundance  in  research  laboratories.  We  need  to  move  on. 

As  we've  discussed  in  this  paper,  reftnrn  can  proceed  akxig  a  number  of  padiways 
(perh^K  in  parallel).  For  instance,  computer  graphics  are  getting  bmter  every  day;  we  can  now 
develop  3-dimenskmal  virtual  environments  where  individuals  can  interact  with  any  artificial 
world  we  dnose  to  program  (or  purduse).  Satellite  transmissions  can  relay  data  to  very  disumt 
locations:  learners  from  difietent  parts  of  die  gltdie  can  access  distal  data,  or  evoi  get  tog^ier 
and  jointly  eiqierienoe  and  solve  various  ptoUems.  Cognitive  tools  abound  (e.g.,  simulators, 
hypertext/hypetmedia  formats,  etc.),  and  we  seem  to  be  ready  to  recast  our  cemviedons  about 
ITS.  Rather  than  trying  to  create  all-knowing,  all-purpose  teaching  machines,  a  more  fruitful 
igipioachmr^  be  fodeveh^  specific  cranputerized  tools.  These  tools  can  be  specific  for  a  given 
domain,  or  general  purpose,  apfdicaUe  across  dmnains.  To  paraphrase  a  well-known  quotadrm: 

A  person  who  is  given  a  fish  will  eat  for  a  day,  but  a  person  who  learns  how  to  fish  will  eat  for  a 
Itfetime. 

We  can  see  the  seeds  of  disermtent  growing.  GotoanylTS-relatedccHifetenoeand 
notice  how  teseatdiets  in  the  field  have  b^pm  to  discontinue  using  die  term  TTS."  Instead,  in  a 
show  of  semandc  squhming,  they  refer  to  advanced  autmnated  instrucdonal  systems  (formerly, 
ITS)  as:  Interactive  riming  Environmertts,  Cogiddve  T)ittns,  Indivkhialized  Teaching  Systems, 
Computer-Assisted  Learning,  Automated  Instrucdmal  Siqiport  Systems,  Cmnputer-Based 
I  canning  Environments,  Immersive  'nitoring  Systmns,  Knowledge  Communications  Systems, 
Computer  Tools,  and  so  on. 

Not  rmly  is  the  ITS  construct  too  ambitious,  but  there  is  no  universally-accepted 
dffinitii^  of  what  comprises  computer  intelligence.  While  our  working  definition  of  intelligence 
is  fairly  qiecific,  diere  exists  a  wide  range  of  criteria  in  the  literature  related  to  emnputer-tutor 
intwiiigfiiftft  Fdr  instance,  some  say  that  for  an  automated  iiKtructional  system  to  earn  the  labd 
"imelUgent,"  it  must  demonstrate  die  ability  to  leam  by  dwwing  an  evolving  knowledge  base. 
Yazdani  and  Lawfer  (1986)  asserted,  "No  system  whidi  is  too  rigid  to  leam  ^uld  be  called 
intelUgmit"  (p.  201).  Odwis  have  argued  diat  imeuigeot  systems  must  provide  for  learner 
crmtrol  during  die  teaming  process  (Papert  1980;  Scardamalia,  et  al.,  1989).  Still  others  (e.g., 
MacKenzie,  1990)  suggest  diat  we  reserve  the  word  "intelligent"  to  describe  only  those  systems 
showing  truly  impressive  advances  (e.g.,  intiiition,  emp«hy).  Are  dwse  even  realistic  goals? 

The  of  AI,  psychology,  and  education  have  all  gready  benefited  fnm  ITS  research. 
But  to  (see  Riture  5),  much  more  systematk  research  is  needed  to  adiieve  some  of  die 

great  potential  offered  by  diese  systems.  One  suggestion  is  to  begin  a  coordinated  stream  of 
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systematic  ITS  research  and  development,  altering  specific  features  of  existing  systems  and 
evaluating  the  results  of  those  changes  in  accordance  with  a  principled  approadt  According  to 
Self  (1989),  "Once  a  sounder  foundation  for  ITS  has  been  q)ecified,  it  becmnes  posaUe  to 
identify  the  elements  of  a  theory  of  ITS.  These  elements  lie  within  (formal)  AI.  in  areas  such  as 
belief  logics,  reasrni  maintenance,  meta-level  architectures,  and  discourse  models-areas  from 
which  ITS  research  has  been  divorced"  (p.  244).  Intelligent  tutoring  systems,  as  we  now  Imow 
them,  may  not  exist  20  years  from  now,  but  we're  (m  the  right  patti,  the  motives  are 
ocunmendaUe,  and  foe  learner  will  ultimately  {uofit 

As  we  began  this  review  of  ITS  with  the  evolution  of  conputer  technology,  so  do  we  end 
it  ITS  and  related,  developing  techncdogies  for  education  and  training  are  constrained  by  two 
in^mrtant  factors:  (a)  the  cost  and  power  of  conqxtters,  and  (b)  foe  pragmatic  and  theoretical 
knowledge  of  how  best  to  employ  foem.  Every  month,  computers  are  dramatically  deoeasing  in 
cost  and  increasing  in  power:  these  changes  bearing  direcdy  on  consumer  knowledge  and 
apidication  of  foe  techncdogy.  While  discusskm  of  foe  interaction  between  foese  two  factors  goes 
beyond  foe  scope  of  this  charter,  we  can  make  straightforward  predictions  about  upcoming 
hardware  and  software  developments.  The  MIPS  (millitms  of  instructions  per  second)  curve  is 
already  converging  on  a  BDPS  (billions  of  instructions  per  second)  curve  in  an  exponential 
explosion  that  knows  no  limits.  Desktop  computers  wifo  1(X)  MIPS  are  currently  availatde,  and 
this  taw  horsepower  makes  a  qualitative  difference  in  computing  possiUlities.  Soon,  powerful 
systems  will  be  availaUe  in  notrtxxdi-  and  calculator-sized  formats  that  fit  into  our  hands,  shirt 
podrets,  and  purses.  Further,  software  tools  mudde  us  to  learn  from,  and  perfonn  within,  all 
major  domains,  such  as  algdxa,  tdology,  physics,  art  history,  computer  science,  hrane 
ecmiomics,  psychology,  botany,  calculus,  accounting,  and  even  manufacturing,  medicitK,  and 
engineerit^.  At  our  finger  t^,  we  will  be  able  to  retrieve  information,  translate  foreign 
languages,  comidete  our  tax  rdums,  work  out  investment  portfolios,  analyze  sales  trends,  and  so 
forth.  Software  will  be  every^ifoere  wifo  embedded  "assistoits"  to  exfdain,  critique,  provide  on¬ 
line  sunxnt  and  coaching,  and  perform  all  of  the  ITS  activities  outlined  in  this  cluqiter.  Society 
stands  at  foe  edge  of  an  this.  Although  foe  timeline  for  foese  exciting  developments  is  uncertain, 
we  do  know  foat  the  researdi  (xxidttclBd  so  fsr  is  just  a  drizzle  in  conqMuison  with  foe  deluge  to 
come. 
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